activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: tmp-storage cleanup with fileQueueCursor
Date Wed, 29 Aug 2012 10:36:00 GMT
thanks for reporting back and closing the loop on this one :-)

On 28 August 2012 22:55, Jeremy Levy <jelevy@gmail.com> wrote:
> I was able to solve both issues, It wasn't clear to me that
> memoryLimit="2mb" was the corresponding setting on the queue policy in
> regards to the memory limit threshold before spooling to disk.  Therefore I
> think it was starting with the disk right away.  After changing that to
> something reasonably higher it now works as expected for both cleaning up
> the disk and spooling.
>
> Hope someone else finds this useful and saves them the trouble.
>
> Jeremy
>
>
>
> On Tue, Aug 28, 2012 at 12:08 PM, Jeremy Levy <jelevy@gmail.com> wrote:
>
>> Also, it doesn't seem as though fileQueueCursor is working as described in
>> many places in the documentation:
>>
>> http://activemq.apache.org/message-cursors.html says:
>> "When memory in the broker reaches its limit, it can page messages to
>> temporary files on disk. "
>>
>> http://activemq.apache.org/javalangoutofmemory.html says:
>> "The Broker (using Message Cursors<http://activemq.apache.org/message-cursors.html>)
>> will spool non-persistent messages to disk when the default memory usage
>> threshold for a destination is reached. This threshold value is specified
>> to the Broker via the <memoryUsage> section of the <systemUsage>
>> configuration in activemq.xml<http://activemq.apache.org/xml-configuration.html>.
>> This feature allows producers to continue sending messages when there are
>> slow consumers without exhausting available memory or reverting to producer
>> flow control <http://activemq.apache.org/producer-flow-control.html>."
>>
>> I've tested this with one queue, configured in with <fileQueueCursor/> and
>> various <memoryUsage> configurations.  Messages appear to always be
>> written to disk.  This, combined with the tmp-storage not being clean up
>> means that my activemq server is a ticking time bomb until it runs out of
>> disk space.
>>
>> I could get away with the disk space not being cleaned out if it was only
>> writing to disk once the memory limit was reached but that doesn't seem to
>> be the case. Any suggestions?
>>
>> Jeremy
>>
>>  On Tue, Aug 28, 2012 at 11:14 AM, Jeremy Levy <jelevy@gmail.com> wrote:
>>
>>> Hi Gary-
>>>
>>> Thanks for the quick response, as I read that it's fixed in 5.6.0,
>>> however I'm using 5.6.0... I'm only seeing the directory cleaned up during
>>> a restart.
>>>
>>> Jeremy
>>>
>>> On Tue, Aug 28, 2012 at 11:03 AM, Gary Tully <gary.tully@gmail.com>wrote:
>>>
>>>> If you are sending non persistent messages see
>>>> https://issues.apache.org/jira/browse/AMQ-3780
>>>>
>>>> On 28 August 2012 15:44, Jeremy Levy <jelevy@gmail.com> wrote:
>>>> > When using fileQueueCursor, as I understand it the data is written to
>>>> the
>>>> > tmp-storage directory.  When does this directory get cleaned up.  For
>>>> > example I tested putting about 3 million messages on a queue and it
>>>> > successfully wrote them to disk (as opposed to memory).  It took up
>>>> about
>>>> > 8.7GB.  I then deleted the queue from the web console, however the
>>>> 8.7GB
>>>> > remained.  Is there a job or something that I need to configure for
>>>> this
>>>> > space to be reclaimed?
>>>> >
>>>> > Thanks.
>>>> >
>>>> > Jeremy
>>>> >
>>>> >
>>>> >  <destinationPolicy>
>>>> >             <policyMap>
>>>> >               <policyEntries>
>>>> >                 <policyEntry topic=">" producerFlowControl="false"
>>>> > memoryLimit="2mb">
>>>> >                   <pendingSubscriberPolicy>
>>>> >                     <vmCursor />
>>>> >                   </pendingSubscriberPolicy>
>>>> >                 </policyEntry>
>>>> >
>>>> >                 <policyEntry queue=">" producerFlowControl="false"
>>>> > memoryLimit="2mb" prioritizedMessages="true">
>>>> >                   <!-- Use VM cursor for better latency
>>>> >                        For more information, see:
>>>> >
>>>> >                        http://activemq.apache.org/message-cursors.html
>>>> >                    -->
>>>> >                   <pendingQueuePolicy>
>>>> >                     <!--vmQueueCursor/-->
>>>> >                     <fileQueueCursor/>
>>>> >                   </pendingQueuePolicy>
>>>> >                 </policyEntry>
>>>> >
>>>> >               </policyEntries>
>>>> >             </policyMap>
>>>> >         </destinationPolicy>
>>>> >
>>>> >
>>>> > --
>>>> > Jeremy Levy
>>>>
>>>>
>>>>
>>>> --
>>>> http://fusesource.com
>>>> http://blog.garytully.com
>>>>
>>>
>>>
>>>
>>> --
>>> Jeremy Levy
>>>
>>
>>
>>
>> --
>> Jeremy Levy
>>
>
>
>
> --
> Jeremy Levy



-- 
http://fusesource.com
http://blog.garytully.com

Mime
View raw message