activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alisenberg <lisenb...@gmail.com>
Subject RE: KahaDB data files not cleaned up
Date Wed, 28 Apr 2010 16:59:04 GMT


Daniel Kluesing-2 wrote:
> 
> I'd be interested as well. 
> 
> I'm not using TTL, so I assumed my orphaned/non-sequence log files were
> due to out of order message consumption, but I also see the occasional log
> file hang around longer than expected. (say db-50.log then next is
> db-400.log, so big gap)
> 
> -----Original Message-----
> From: B.D. [mailto:boris.dainson@gmail.com] 
> Sent: Monday, February 08, 2010 8:39 AM
> To: users@activemq.apache.org
> Subject: Re: KahaDB data files not cleaned up
> 
> 
> Any suggestions on how to prevent pileups of data/kahadb/db-xxx.log files? 
> 
> I just had to clean up 301 data/kahadb/db-xxx.log files that accumulated
> on
> a QA machine over 6 days. Message time to live is one day so that should
> not
> be happening. 
> 
> Logs show KahaDB cleanup was taking longer than expected: 
> activemq.log:2010-02-08 13:31:29,664 | INFO  | Slow KahaDB access: cleanup
> took 591 | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ
> Journal Checkpoint Worker
> 
> 
> Thanks
> 
> 
> 
> B.D. wrote:
>> 
>> Hi,
>> 
>> I’m using ActiveMQ 5.3 with KahaDB. Two brokers are in a network with
>> failover. Publishers are sending messages to a composite destination that
>> forwards to a topic or a queue.  It works ok with topics but with queues
>> after a while there is accumulation of db-xxx.log files in the
>> data/kahadb
>> directory. 
>> 
>> Files will be several days old despite the fact that all messages have a
>> one-day time to live. Dead letter strategy is
>> individualDeadLetterStrategy
>> with processExpired="false" so I do not expect expired messages to stay
>> around in any form. Purging queues using admin console does not get rid
>> of
>> these files. The only way I found to clean it up was to stop the broker
>> and clean up the whole data directory. 
>> 
>> I could reproduce this problem in a test setup by starting/stopping
>> brokers, publishers and subscribers. Older data/kahadb/db-X.log files are
>> normally getting deleted but some are not, see below. In these tests
>> time-to-live on all messages was 10 minutes so this should not be
>> happening. See directory lists below.
>> 
>> With a single broker I could not reproduce that same problem. Older
>> db-X.log files were always eventually getting cleaned up. However I
>> noticed that db.data file can grow. It reached 360M while it normally
>> stays below 34M.
>> 
>> Any suggestion how to prevent growth of files in the data/kahadb
>> directory? 
>> 
>> Thanks,
>> 
>> 
>> Broker 1 in a 2-broker network: 
>> 33555255	2010-01-07	23:18:13	data/kahadb/db-80.log
>> 33030144	2010-01-08	07:46:16	data/kahadb/db-177.log
>> 33554509	2010-01-07	22:50:21	data/kahadb/db-74.log
>> 33555184	2010-01-08	01:34:55	data/kahadb/db-110.log
>> 33554441	2010-01-07	22:41:00	data/kahadb/db-72.log
>> 33554519	2010-01-08	04:40:53	data/kahadb/db-144.log
>> 33554522	2010-01-07	19:51:33	data/kahadb/db-34.log
>> 33555005	2010-01-08	04:09:10	data/kahadb/db-138.log
>> 33555019	2010-01-08	02:07:52	data/kahadb/db-117.log
>> 33554494	2010-01-07	18:52:05	data/kahadb/db-21.log
>> 33555122	2010-01-08	04:30:02	data/kahadb/db-142.log
>> 3287296	2010-01-08	07:46:16	data/kahadb/db.redo
>> 33554733	2010-01-08	00:40:15	data/kahadb/db-98.log
>> 33554703	2010-01-08	07:34:38	data/kahadb/db-175.log
>> 33554997	2010-01-08	07:26:41	data/kahadb/db-174.log
>> 10763	2010-01-08	07:46:16	data/kahadb/db.free
>> 33555298	2010-01-07	23:04:18	data/kahadb/db-77.log
>> 33554537	2010-01-07	23:45:26	data/kahadb/db-86.log
>> 23187456	2010-01-08	07:46:16	data/kahadb/db.data
>> 33555022	2010-01-08	02:52:40	data/kahadb/db-125.log
>> 
>> 
>> Broker 2 in a 2-broker network: 
>> 33030144	2010-01-08	07:46:15	data/kahadb/db-172.log
>> 1756504	2010-01-08	12:07:34	data/kahadb/db.redo
>> 33554527	2010-01-07	18:29:16	data/kahadb/db-3.log
>> 33554595	2010-01-07	17:49:28	data/kahadb/db-1.log
>> 0		2010-01-07	16:25:00	data/kahadb/lock
>> 2543616	2010-01-08	12:07:34	data/kahadb/db.data
>> 33554596	2010-01-07	23:56:35	data/kahadb/db-79.log
>> 
>> 
>> Single broker (not a network): 
>> 33030144 	Jan 14 18:25 db-1318.log
>> 364490752 	Feb  3 10:22 db.data
>> 3287296 	Feb  3 10:22 db.redo
>> 0 		Jan  8 12:33 lock
>> 
>> 
> 
> -- 
> View this message in context:
> http://old.nabble.com/KahaDB-data-files-not-cleaned-up-tp27441942p27502591.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> 
> 
> 

I have the same problem...
-- 
View this message in context: http://old.nabble.com/KahaDB-data-files-not-cleaned-up-tp27441942p28391322.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message