activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <>
Subject Re: Large number of queues (HowTo)
Date Mon, 18 May 2009 20:12:39 GMT
Awesome! - thanks for getting this info together! will put this on the  
Apache site

On 18 May 2009, at 19:27, Jose Luna wrote:

> Hello,
> We have the requirement of creating a very large number of queues  
> (tens of thousands) in a single broker.  My searches turned up  
> several others trying to do something similar, but with no results.  
> We were recently able to accomplish this, so I decided to write a  
> little howto. All of this information can be found in the mailing  
> list, wiki, or XML reference, but it wasn't easy to pull it all  
> together.  Hopefully this will help someone.  Also, please let me  
> know if there are any inaccuracies, or if anything can be added.
> == Thread Count ==
> === "QueueThread" ===
> One of the issues that you'll come across is receiving the out of  
> memory error "unable to create new native thread" when you create  
> thousands of queues.  A "QueueThread" thread is created for every  
> new queue.  The optimizedDispatch property for the queue policyEntry  
> seemed to address our needs, the description states "don't use a  
> separate thread for dispatching from a Queue"  (

> ). However, turning this on just seemed to create "TempQueue"  
> threads instead, still creating a thread for each queue.  Digging a  
> little deeper, we find that using OptimizedDispatch creates the  
> thread using a TaskRunner, which should pool the threads.   After  
> further research, we found that the property  
> org.apache.activemq.UseDedicatedTaskRunner should be set to false so  
> that the task runner actually pools the threads (

>  --- this document also provides other useful info, like
> how to lower the stack size for each thread).
> Example Policy Entry in INSTALL_DIR/conf/activemq.conf:
> <policyEntries>
>    <policyEntry queue=">" memoryLimit="15mb"  
> optimizedDispatch='true'/>
> </policyEntries>
> You can either change the 'UseDedicatedTaskRunner' property in your  
> startup script, INSTALL_DIR/bin/activemq:
> ACTIVEMQ_OPTS="-Xmx512M - 
> Dorg.apache.activemq.UseDedicatedTaskRunner=false"
> Or you can set ACTIVEMQ_OPTS in /etc/activemq.conf .
> Note: It would be very useful it the documentation listed above  
> referenced each other, as they are most useful when used together.
> === "Checkpoint" thread ===
> If you're using persistent storage, a "Checkpoint" thread is also  
> created for every queue that is created. These are used to write  
> persistent messages to the store.  Fortunately this is already  
> created with a task runner, so setting the UseDedicatedTaskRunner  
> property to false will prevent these thread from overhwelming your  
> system.
> === Connection related threads ===
> If you're also dealing with high thread count due to a very high  
> number of connections, see

> .  Also, consider reducing the stack size for each thread (the JVM  
> option -Xss).
> == "too many open files" ==
> ActiveMQ uses the amqPersistenceAdapter by default for persistent  
> messages.  Unfortunately, this persistence adapter (as well as the  
> kahaPersistenceAdapter) opens a file descriptor for each queue.   
> When creating large numbers of queues, you'll quickly run into the  
> limit for your OS.   There are two possible solutions to this:  1.)  
> raise the per-process open file limit for your OS 2.) Use a jdbc  
> persistent adapter.
> The first can simply be googled for your OS.  The second uses a  
> connection pool to your DB of choice, and no longer requires an open  
> file for each queue created.
> An unrelated issue is running out of file descriptors because there  
> are too many connections.  (In linux, and other OSes, a file  
> descriptor is used for open sockets).  I don't know of any other way  
> to handle that, other than increasing the limit for your process.
> == Conclusion ==
> We were able to create more than 20k queues using ~30 threads in a  
> test environment with a 512MB heap.  Environment: ActiveMQ 5.2.0,  
> SUN Java 1.6.0, Ubuntu 9.04 .  Our testing hasn't been exhaustive,  
> but everything is running smoothly so far.
> JLuna

View raw message