activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin C." <mart...@gmx.at>
Subject Re: KahaDB storage size grows despite almost no pending messages
Date Fri, 10 Jun 2011 22:45:15 GMT
Hi,

I tried logging and various approaches. I even deleted all queues,
shut down cleanly via JMX, restarted several times. Except a long
startup for checksumming and verifying the logs, only checkpoint
entries. I even deleted the metadata store in order to force recovery
from the logfile, but still, no log file gets deleted, which in my
case remains at a 67% usage of a 10GB limit, i.e. 6.7GB logs.

I am not that deep into development of ActiveMQ/KahaDB, so a pointer
where I should set a breakpoint in the debugger in order to find
if/why KahaDB believes the log files are still needed would be
helpful.

Thanks for all your input & help!

Best regards,
Martin

On Wed, Jun 8, 2011 at 10:40 AM, James Green <james.mk.green@gmail.com> wrote:
> I experienced the same. To debug add the following to your log4j.properties
> file (if needed):
>
> log4j.appender.kahadb=org.apache.log4j.RollingFileAppender
> log4j.appender.kahadb.file=${activemq.base}/data/kahadb.log
> log4j.appender.kahadb.maxFileSize=1024KB
> log4j.appender.kahadb.maxBackupIndex=5
> log4j.appender.kahadb.append=true
> log4j.appender.kahadb.layout=org.apache.log4j.PatternLayout
> log4j.appender.kahadb.layout.ConversionPattern=%d [%-15.15t] %-5p
> %-30.30c{1} - %m%n
> log4j.logger.org.apache.activemq.store.kahadb.MessageDatabase=TRACE, kahadb
>
> Restart AMQ and let the pruner process run (give it a minute or two for
> example).
>
> Examine the log file and look for pruning of the queues. It gives the queue
> and the log file numbers that remain candidates for removal.
>
> At some point you'll hit a queue and the number of logs will suddenly drop
> unexpectedly. That queue has likely got messages that AMQ is holding a
> reference to.
>
> In our case we had NC_ durable clients that had long been disposed of by
> ourselves. After deleting these the subsequent expiring / purging run of
> kahadb removed the files and freed up literally gigs of space.
>
> Hope this helps. It should really be documented as a FAQ!
>
> James
>
> On 4 June 2011 19:38, Martin C. <martinc@gmx.at> wrote:
>
>> Hi,
>>
>> I experience the problem that over time my KahaDB storage fills up
>> despite almost no queue showing pending messages. I upgraded from
>> ActiveMQ 5.4.2 to 5.5.0 because I thought that according to the change
>> log these leaks in KahaDB should be fixed in 5.5.0, but for me it
>> seems they aren't. What can I do to diagnose the problem further?
>>
>> I'll try if restarting the broker alone is enough to clean the store,
>> but I'll have to wait for our next maintenance interval before I can
>> try this.
>>
>> Best regards,
>> Martin
>>
>

Mime
View raw message