activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: KahaDB - log f
Date Tue, 14 Sep 2010 17:40:20 GMT
You understand correctly, only unreferenced files are reclaimed. Often
using smaller journal data files makes cleanup more immediate (as the
chance of a slow message being in every data file decreases) but in
your case it may makes sense to increase the max data files if you
cannot increase the max number of ope files.
What is the current open file limit?
I think it may be worth a test with a smaller data file size just to be sure.

Unfortunately there is no way to have a store per queue at the moment
or partition the queues across stores. That would not be that hard to
implement though and has come up before.  What is needed is an
alternative DestinationFactoryImpl.

On 14 September 2010 18:25, nervousbadger <nervousbadger@googlemail.com> wrote:
>
> Hi,
>
> We have the following scenario:
>
> Single broker with ~20 queues. Some queues have extremely high throughput
> (thousands of messages per second produced and consumed). Other queues have
> low throughput and messages may remain on these queues for a couple of hours
> before being consumed. All messages are persistent.
>
> We are running out of file handles for the activemq process - looking in the
> kahadb directory we can see ~1000 db-xxx.log files. If I understand the
> documentation correctly, then each individual log file cannot be removed
> until there are no more references to the messages logged within.
>
> I think this means that for each 32MB file, if there is a single message
> logged in it that has been delivered to one of our slow throughput queues
> then even though every other message logged in the file has been delivered,
> the file cannot be deleted until the final message has been delivered.
>
> If this is correct, is there any way around this (such as having the log
> files be per queue)? Or will we have to increase the maximum log file
> size/maximum number of open files?
>
> Thanks,
> Pete.
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/KahaDB-log-f-tp2539334p2539334.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Mime
View raw message