activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: ActiveMQ needs tempUsage for recovery of persistent messages on startup?
Date Wed, 16 Nov 2011 12:50:14 GMT
yes.

On 16 November 2011 12:01, Martin C. <martinc@gmx.at> wrote:
> Hi Gary,
>
> thanks for the very fast response. I will change the configuration
> accordingly. So basically you say I should remove the
> <pendingQueuePolicy> as a whole?
>
> Best regards,
> Martin
>
> On Wed, Nov 16, 2011 at 11:34 AM, Gary Tully <gary.tully@gmail.com> wrote:
>> You should not be using the file pending message cursor in this case.
>> The default store cursor is best, it will use a file cursor for non
>> persistent messages and leave excess (exceed the cache) persistent
>> messages in the store.
>> When it is specified as the default cursor in configuration, it is
>> used for both persistent and non persistent messages.
>>
>> The file pending message cursor replays all messages on startup such
>> that it can do an append to the cursor on the next add but it is only
>> intended for really slow stores, where reading from the local file
>> system is faster than reading from the store. This is not the case
>> with kahaDB.
>>
>> On 16 November 2011 10:26, Martin C. <martinc@gmx.at> wrote:
>>> Hi,
>>>
>>> I a have a question regarding fileQueueCursor and systemUsage on
>>> ActiveMQ 5.5.0: Below you see my configuration, which limits the
>>> memory to 64MB, store usage to 10GB and tempUsage to 1GB.
>>>
>>> I have a KahaDB with about 300000 messages, using 330MB in the kahadb
>>> directory, all messages are persistent. If I start ActiveMQ, I see the
>>> tmp_storage directory grow to 1GB, where it finally blocks and
>>> recovery of the messages starts hanging:
>>>
>>> [...]
>>> DEBUG | default:memory: usage change from: 2% of available memory, to:
>>> 1% of available memory
>>> DEBUG | default:memory:queue://local.queue.myqueue:memory: usage
>>> change from: 1% of available memory, to: 0% of available memory
>>> DEBUG | default:memory: usage change from: 1% of available memory, to:
>>> 0% of available memory
>>>  INFO | cursor for queue://local.queue.myqueue has recovered 50000
>>> messages. 15% complete
>>>  INFO | cursor for queue://local.queue.myqueue has recovered 100000
>>> messages. 31% complete
>>>  INFO | cursor for queue://local.queue.myqueue has recovered 150000
>>> messages. 47% complete
>>>  INFO | cursor for queue://local.queue.myqueue has recovered 200000
>>> messages. 62% complete
>>>
>>> The memory usage goes up to around 75%, where it starts dropping down
>>> to the 0% from the logfile above. This is where it keeps hanging, with
>>> the tmp_storage directory at 1GB diskspace.
>>>
>>> If I increase the tempUsage to 10GB, the startup works fine
>>> (tmp_storage becomes about 1.4GB in this case). Can you explain to me,
>>> why I need to increase tempUsage in order to restart my broker with a
>>> lot of persistent messages left? As far as I understood the tempUsage
>>> is for storing non-persistent messages, persistent messages should be
>>> affected by storeUsage?
>>>
>>> Thanks in advance!
>>>
>>> Best regards,
>>> Martin
>>>
>>> PS: the relevant parts from my configuration
>>>
>>>  <broker xmlns="http://activemq.apache.org/schema/core"
>>> brokerName="localhost" dataDirectory="${activemq.base}/data"
>>>    populateJMSXUserID="true" useJmx="true">
>>>
>>>    <destinationPolicy>
>>>      <policyMap>
>>>        <policyEntries>
>>>          <policyEntry topic=">" producerFlowControl="false"
>>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="60000">
>>>              <vmCursor />
>>>          </policyEntry>
>>>          <policyEntry queue=">" producerFlowControl="false"
>>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="86400000">
>>>            <deadLetterStrategy>
>>>              <individualDeadLetterStrategy queuePrefix="DLQ."
>>> useQueueForQueueMessages="true"
>>>                processExpired="true" processNonPersistent="false" />
>>>            </deadLetterStrategy>
>>>            <pendingQueuePolicy>
>>>              <fileQueueCursor />
>>>            </pendingQueuePolicy>
>>>          </policyEntry>
>>>        </policyEntries>
>>>      </policyMap>
>>>    </destinationPolicy>
>>>
>>>    <persistenceAdapter>
>>>    <kahaDB directory="${activemq.base}/data/kahadb"
>>>         checkForCorruptJournalFiles="true" checksumJournalFiles="true"
>>>         concurrentStoreAndDispatchTopics="false"
>>>         concurrentStoreAndDispatchQueues="false"
>>>         />
>>>    </persistenceAdapter>
>>>
>>>    <systemUsage>
>>>      <systemUsage sendFailIfNoSpace="true">
>>>        <memoryUsage>
>>>          <memoryUsage limit="64 mb" />
>>>        </memoryUsage>
>>>        <storeUsage>
>>>          <storeUsage limit="10 gb" />
>>>        </storeUsage>
>>>        <tempUsage>
>>>          <tempUsage limit="1 gb" />
>>>        </tempUsage>
>>>      </systemUsage>
>>>    </systemUsage>
>>>  </broker>
>>>
>>
>>
>>
>> --
>> http://fusesource.com
>> http://blog.garytully.com
>>
>



-- 
http://fusesource.com
http://blog.garytully.com

Mime
View raw message