activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin C." <mart...@gmx.at>
Subject Re: ActiveMQ needs tempUsage for recovery of persistent messages on startup?
Date Wed, 16 Nov 2011 12:01:21 GMT
Hi Gary,

thanks for the very fast response. I will change the configuration
accordingly. So basically you say I should remove the
<pendingQueuePolicy> as a whole?

Best regards,
Martin

On Wed, Nov 16, 2011 at 11:34 AM, Gary Tully <gary.tully@gmail.com> wrote:
> You should not be using the file pending message cursor in this case.
> The default store cursor is best, it will use a file cursor for non
> persistent messages and leave excess (exceed the cache) persistent
> messages in the store.
> When it is specified as the default cursor in configuration, it is
> used for both persistent and non persistent messages.
>
> The file pending message cursor replays all messages on startup such
> that it can do an append to the cursor on the next add but it is only
> intended for really slow stores, where reading from the local file
> system is faster than reading from the store. This is not the case
> with kahaDB.
>
> On 16 November 2011 10:26, Martin C. <martinc@gmx.at> wrote:
>> Hi,
>>
>> I a have a question regarding fileQueueCursor and systemUsage on
>> ActiveMQ 5.5.0: Below you see my configuration, which limits the
>> memory to 64MB, store usage to 10GB and tempUsage to 1GB.
>>
>> I have a KahaDB with about 300000 messages, using 330MB in the kahadb
>> directory, all messages are persistent. If I start ActiveMQ, I see the
>> tmp_storage directory grow to 1GB, where it finally blocks and
>> recovery of the messages starts hanging:
>>
>> [...]
>> DEBUG | default:memory: usage change from: 2% of available memory, to:
>> 1% of available memory
>> DEBUG | default:memory:queue://local.queue.myqueue:memory: usage
>> change from: 1% of available memory, to: 0% of available memory
>> DEBUG | default:memory: usage change from: 1% of available memory, to:
>> 0% of available memory
>>  INFO | cursor for queue://local.queue.myqueue has recovered 50000
>> messages. 15% complete
>>  INFO | cursor for queue://local.queue.myqueue has recovered 100000
>> messages. 31% complete
>>  INFO | cursor for queue://local.queue.myqueue has recovered 150000
>> messages. 47% complete
>>  INFO | cursor for queue://local.queue.myqueue has recovered 200000
>> messages. 62% complete
>>
>> The memory usage goes up to around 75%, where it starts dropping down
>> to the 0% from the logfile above. This is where it keeps hanging, with
>> the tmp_storage directory at 1GB diskspace.
>>
>> If I increase the tempUsage to 10GB, the startup works fine
>> (tmp_storage becomes about 1.4GB in this case). Can you explain to me,
>> why I need to increase tempUsage in order to restart my broker with a
>> lot of persistent messages left? As far as I understood the tempUsage
>> is for storing non-persistent messages, persistent messages should be
>> affected by storeUsage?
>>
>> Thanks in advance!
>>
>> Best regards,
>> Martin
>>
>> PS: the relevant parts from my configuration
>>
>>  <broker xmlns="http://activemq.apache.org/schema/core"
>> brokerName="localhost" dataDirectory="${activemq.base}/data"
>>    populateJMSXUserID="true" useJmx="true">
>>
>>    <destinationPolicy>
>>      <policyMap>
>>        <policyEntries>
>>          <policyEntry topic=">" producerFlowControl="false"
>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="60000">
>>              <vmCursor />
>>          </policyEntry>
>>          <policyEntry queue=">" producerFlowControl="false"
>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="86400000">
>>            <deadLetterStrategy>
>>              <individualDeadLetterStrategy queuePrefix="DLQ."
>> useQueueForQueueMessages="true"
>>                processExpired="true" processNonPersistent="false" />
>>            </deadLetterStrategy>
>>            <pendingQueuePolicy>
>>              <fileQueueCursor />
>>            </pendingQueuePolicy>
>>          </policyEntry>
>>        </policyEntries>
>>      </policyMap>
>>    </destinationPolicy>
>>
>>    <persistenceAdapter>
>>    <kahaDB directory="${activemq.base}/data/kahadb"
>>         checkForCorruptJournalFiles="true" checksumJournalFiles="true"
>>         concurrentStoreAndDispatchTopics="false"
>>         concurrentStoreAndDispatchQueues="false"
>>         />
>>    </persistenceAdapter>
>>
>>    <systemUsage>
>>      <systemUsage sendFailIfNoSpace="true">
>>        <memoryUsage>
>>          <memoryUsage limit="64 mb" />
>>        </memoryUsage>
>>        <storeUsage>
>>          <storeUsage limit="10 gb" />
>>        </storeUsage>
>>        <tempUsage>
>>          <tempUsage limit="1 gb" />
>>        </tempUsage>
>>      </systemUsage>
>>    </systemUsage>
>>  </broker>
>>
>
>
>
> --
> http://fusesource.com
> http://blog.garytully.com
>

Mime
View raw message