activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <>
Subject Re: Poort ActiveMQ Persistence Performance
Date Sat, 29 Sep 2007 06:11:30 GMT

Hi Darrell,

Derby is the bottleneck for the Journal, in version 4.x by default  
the entire message will be moved from the journal into the store (by  
default Derby) - which is slow. In reality relational databases  
aren't well suited to streaming queued data through. Kaha persistence  
on the overhand doesn't use a journal, and is optimized for streaming  
queued data. However, the big difference is in reality down to the  
fact that Kaha doesn't ensure data is synced to disk between writes  
by default.

For version 5 of activemq we've made big changes to the way messages  
are passed through the broker. In version 4, references to the  
messages in store were usually held in memory, which enabled  
reasonable performance, meant the code was easier to maintain but  
ultimately meant that activemq could not scale to  coping with queue  
depths of 100s of millions of messages without requiring a very large  
memory footprint to hold those message references!

For version 5, message references are no longer held, but messages  
are instead paged from the store when required. Its relatively  
straight forward to pull messages from a queue when they are  
required, but that is generally slow. So we've catered for a hybrid  
push/pull model: We push (as in version 4) as long as the consumers  
can keep up, but if they can't - we swap back to pulling out of the  
store. This enables persistent messaging to have very good  
performance when the consumers aren't slow, but also enables scaling  

However, having done this, the requirement on the message store is  
more demanding than version 4.  So the other big change we've done  
for version 5 is to change the default messaging to use the journal  
with kaha for the default messaging. We only need to hold references  
in kaha - pointing to the actual message in the journal. This  
combination is actually faster and more resilient than kaha in  
version 4 - as if the reference store (kaha by default) is corrupted,  
we can generate it again from the journal.

So long story short - use the default message store (AMQ) in version 5!



On Sep 28, 2007, at 7:00 PM, DarrellFuller wrote:

> Our system is taking 6~7 seconds to pull a message off of a queue  
> using a
> simple selector (id = ?).  The queue has about 20,000 messages  
> which are
> fairly small.  I ran through a quick series of tests and this is  
> what I came
> up with.
> # of Messages    = 20,000
> Avg time to Send = 64 ms  (per send)    // Using a message converter
> Avg time to read = 16 ms   (per read)     // Read 500 messages of  
> queue of
> 20,000 messages
> ** Default Journaling with Derby **
> # of Messages    = 20,000
> Avg time to Send = Took over 20 minutes for test.  Appox 1 min per  
> message
> Avg time to read = Aborted Test after 30 minutes.  Wasn't worth  
> completing.
> ** Using Kaha Persistence **
> # of Messages    = 20,000
> Avg time to Send = ~ 500 ms  (per send)
> Avg time to read =  ~ 1.2 seconds  (per read)
> My question is why is the default persistence appear to be SO SLOW...
> Especially compared to the Kaha persistence.
> Is Derby the bottleneck with the default journaling?
> If not what is the difference between the standard Journaling and  
> Kaha?
> Any guidance would be appreciated..
> Thank you,
> Darrell
> -- 
> View this message in context: 
> Persistence-Performance-tf4535973s2354.html#a12945570
> Sent from the ActiveMQ - User mailing list archive at

View raw message