activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Kluesing ...@bluekai.com>
Subject RE: KahaDB data files not cleaned up
Date Mon, 08 Feb 2010 18:26:06 GMT
I'd be interested as well. 

I'm not using TTL, so I assumed my orphaned/non-sequence log files were due to out of order
message consumption, but I also see the occasional log file hang around longer than expected.
(say db-50.log then next is db-400.log, so big gap)

-----Original Message-----
From: B.D. [mailto:boris.dainson@gmail.com] 
Sent: Monday, February 08, 2010 8:39 AM
To: users@activemq.apache.org
Subject: Re: KahaDB data files not cleaned up


Any suggestions on how to prevent pileups of data/kahadb/db-xxx.log files? 

I just had to clean up 301 data/kahadb/db-xxx.log files that accumulated on
a QA machine over 6 days. Message time to live is one day so that should not
be happening. 

Logs show KahaDB cleanup was taking longer than expected: 
activemq.log:2010-02-08 13:31:29,664 | INFO  | Slow KahaDB access: cleanup
took 591 | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ
Journal Checkpoint Worker


Thanks



B.D. wrote:
> 
> Hi,
> 
> I’m using ActiveMQ 5.3 with KahaDB. Two brokers are in a network with
> failover. Publishers are sending messages to a composite destination that
> forwards to a topic or a queue.  It works ok with topics but with queues
> after a while there is accumulation of db-xxx.log files in the data/kahadb
> directory. 
> 
> Files will be several days old despite the fact that all messages have a
> one-day time to live. Dead letter strategy is individualDeadLetterStrategy
> with processExpired="false" so I do not expect expired messages to stay
> around in any form. Purging queues using admin console does not get rid of
> these files. The only way I found to clean it up was to stop the broker
> and clean up the whole data directory. 
> 
> I could reproduce this problem in a test setup by starting/stopping
> brokers, publishers and subscribers. Older data/kahadb/db-X.log files are
> normally getting deleted but some are not, see below. In these tests
> time-to-live on all messages was 10 minutes so this should not be
> happening. See directory lists below.
> 
> With a single broker I could not reproduce that same problem. Older
> db-X.log files were always eventually getting cleaned up. However I
> noticed that db.data file can grow. It reached 360M while it normally
> stays below 34M.
> 
> Any suggestion how to prevent growth of files in the data/kahadb
> directory? 
> 
> Thanks,
> 
> 
> Broker 1 in a 2-broker network: 
> 33555255	2010-01-07	23:18:13	data/kahadb/db-80.log
> 33030144	2010-01-08	07:46:16	data/kahadb/db-177.log
> 33554509	2010-01-07	22:50:21	data/kahadb/db-74.log
> 33555184	2010-01-08	01:34:55	data/kahadb/db-110.log
> 33554441	2010-01-07	22:41:00	data/kahadb/db-72.log
> 33554519	2010-01-08	04:40:53	data/kahadb/db-144.log
> 33554522	2010-01-07	19:51:33	data/kahadb/db-34.log
> 33555005	2010-01-08	04:09:10	data/kahadb/db-138.log
> 33555019	2010-01-08	02:07:52	data/kahadb/db-117.log
> 33554494	2010-01-07	18:52:05	data/kahadb/db-21.log
> 33555122	2010-01-08	04:30:02	data/kahadb/db-142.log
> 3287296	2010-01-08	07:46:16	data/kahadb/db.redo
> 33554733	2010-01-08	00:40:15	data/kahadb/db-98.log
> 33554703	2010-01-08	07:34:38	data/kahadb/db-175.log
> 33554997	2010-01-08	07:26:41	data/kahadb/db-174.log
> 10763	2010-01-08	07:46:16	data/kahadb/db.free
> 33555298	2010-01-07	23:04:18	data/kahadb/db-77.log
> 33554537	2010-01-07	23:45:26	data/kahadb/db-86.log
> 23187456	2010-01-08	07:46:16	data/kahadb/db.data
> 33555022	2010-01-08	02:52:40	data/kahadb/db-125.log
> 
> 
> Broker 2 in a 2-broker network: 
> 33030144	2010-01-08	07:46:15	data/kahadb/db-172.log
> 1756504	2010-01-08	12:07:34	data/kahadb/db.redo
> 33554527	2010-01-07	18:29:16	data/kahadb/db-3.log
> 33554595	2010-01-07	17:49:28	data/kahadb/db-1.log
> 0		2010-01-07	16:25:00	data/kahadb/lock
> 2543616	2010-01-08	12:07:34	data/kahadb/db.data
> 33554596	2010-01-07	23:56:35	data/kahadb/db-79.log
> 
> 
> Single broker (not a network): 
> 33030144 	Jan 14 18:25 db-1318.log
> 364490752 	Feb  3 10:22 db.data
> 3287296 	Feb  3 10:22 db.redo
> 0 		Jan  8 12:33 lock
> 
> 

-- 
View this message in context: http://old.nabble.com/KahaDB-data-files-not-cleaned-up-tp27441942p27502591.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message