activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Mielke <tors...@fusesource.com>
Subject Re: What is the resource footprint of a queue on broker and client sides?
Date Fri, 16 Sep 2011 13:37:23 GMT
Can't get you all the number you ask for but

- since you already set UseDedicatedTaskRunner=false there is no extra thread created for
each queue. Instead a thread pool is used 
- without that setting each queue used its own thread. So memory overhead is that of a new
thread plus surely some more bytes (don't know how much)
- file descriptor wise it should not make a diff as all queues are persisted through one instance
of KahaDB (or whatever configured persistence adapter). 
- disk space: Each newly created queue is persisted to disk. I don't see how that information
alone (no msgs on queue) would take more than a couple of bytes. Its probably just one entry
in the KahaDB journal file

There is a ActiveMQ tuning guide [1] available at www.fusesource.com that might provide additional
tuning tips for your scenario. 
However there will be a limit as to how many queues you can have on one broker. 
Perhaps look into distributing the load across multiple independent brokers.

[1] http://fusesource.com/docs/broker/5.5/tuning/index.html


Torsten Mielke
torsten@fusesource.com
tmielke@blogspot.com



On Sep 16, 2011, at 12:52 AM, vdevadhar wrote:

> Dear Active MQ Users,
> 
> Re-posting this question due to lack of reply; We are seeing 420K memory
> used per queue in the broker; Is that expected? This information is vital in
> our decision making; If any of you know, please respond.
> 
> Thanks
> Vijay
> 
> ------------------------------------------------------------------------------
> 
> Dear Active MQ users group,
> 
> I am benchmarking the maximum number of queues that activemq can support;
> With default config I can
> go up to 1000 queues;
> 
> For more than 1000, I need to add in the -
> xbean:conf/activemq-scalability.xml to the command line for
> broker startup;  
> 
> I also needed to change the Xmx in activemq file to 4096M like shown below.
> ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx4096M"
> 
> Also, I had to add -Dorg.apache.activemq.UseDedicatedTaskRunner=false
> to ACTIVEMQ_OPTS to get the client, server from running out of memory and
> ability to create threads;
> 
> Now, I am able to create 10,000 + queues; Around 11108, I think I see memory
> pressures; At this point,
> I am seeing
> Destination destination = session.createQueue(queueName);
> 
> and/or
> 
>        MessageConsumer consumer = session.createConsumer(destination);
>        consumer.setMessageListener(amqHandler);
> 
> starts to seriously slow down;
> 
> Instead of further increasing JVM memory and further going up the number of
> queues and iterating, I
> wanted to find out the resource footprint required by a queue.  What are the
> number of threads,
> memory, file descriptors, disk space that are needed when a new queue is
> added to active mq?
> Assume empty or almost empty queues. I want this information for both client
> and broker side.
> Assume the above mentioned scalable configs and suggest any other tweaks if
> necessary.
> 
> Use case
> 
> We have 200 or so different job types that come in from 5000 or so
> customers; The more queues we
> can create, the more natural the system behavior since as we put multiple
> job types or customers on the
> same queue, we create sequencing between unrelated things. I know million
> queues don't make a whole lot
> of sense, but I am trying to find a big number that can be scalable.
> 
> Thanks
> Vijay 





Mime
View raw message