activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: confused about Usage Manager Memory Limit reached
Date Mon, 15 Dec 2014 01:04:23 GMT
Sorry for the delay getting back to you.

I'm not clear what delay error messages you're referring to; if you're
still having problems related to them, can you include a sample to help
clarify what you mean?

I got the 70GB by multiplying your 70% JVM percentage by your 100GB store
limit instead of your JVM heap size, which of course doesn't make any
sense.  Sorry for the confusion.
On Dec 9, 2014 8:40 PM, "brian" <brian.ally@conceptual.ca> wrote:

On 14-12-08 08:57 PM, Tim Bain wrote:

> The log line tells you how you're doing against two different limits: the
> per-destination limit and the entire-broker limit.  The per-destination
> limit is first, and the log line says you're using 0% of what you've
> allocated per destination, so you're right that you're fine there.  The
> full-broker limit is second, and the log line says you're using 104% of the
> available memory, which is why you're getting flow controlled: although the
> individual destination is empty, the broker as a whole is full, and this
> one message just happens to be the straw that breaks the camel's back.  So
> figure out what other destination has lots of messages (or large messages)
> to be using 70GB(!) of memory...
>


Thanks for the response. I've spent some time digging into it further and
your remarks helped to clarify some things. The following links helped a
bunch, also:

http://www.christianposta.com/blog/?p=273
http://open-source-adventures.blogspot.ca/2009/01/limiting-
disk-store-usage-with-active.html (contains some deprecated info)
https://access.redhat.com/documentation/en-US/Fuse_
Message_Broker/5.4/html/Using_Persistent_Messages/files/
FuseMBPersistIntro.html

I've since adjusted my config in the following way:

<persistenceAdapter>
  <kahaDB
    directory="${activemq.data}/kahadb"
    journalMaxFileLength="100mb"
    enableJournalDiskSyncs="true"
    checksumJournalFiles="true"
    checkForCorruptJournalFiles="true"
    indexCacheSize="10000"
    cleanupInterval="30000"
    checkpointInterval="5000"
  />
</persistenceAdapter>


<systemUsage>
  <systemUsage>
    <memoryUsage>
      <memoryUsage percentOfJvmHeap="70" />
    </memoryUsage>
    <storeUsage>
      <storeUsage limit="50 gb" />

    </storeUsage>
    <tempUsage>
      <tempUsage limit="50 gb"/>
    </tempUsage>
  </systemUsage>
</systemUsage>


I'd noticed that the log messages mentioned the delay time in increasing
intervals of 30 seconds. Could this be related to cleanupInterval?

I should point out that these message numbers are an infrequent event with
this application. The norm is to pass significantly fewer messages at a
time. With the above changes, i've run this maximal sync several times
without further trouble, save for the last run, in which a single message
refused to leave the queue, and without any suggestion in the log that AMQ
was flow controlling the consumer. I'll keep playing with it.

And did you mean ~700MB? That's much closer to what i'm seeing of the
actual data being sent over the wire.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message