activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hiram Chirino <hi...@hiramchirino.com>
Subject Re: Variable/dynamic prefetch size
Date Mon, 05 Nov 2012 10:01:05 GMT
You can setup per consumer prefetch sizes.  If you can send to a different
queue based on how long each message will take to process (for example
fastqueue and slowqueue)  Then you can setup consumers on fastqueue with a
large prefetch and consumer on slowqueue with a small prefetch.


On Mon, Nov 5, 2012 at 2:22 AM, sdonovan_uk <sdonovan_uk@yahoo.com> wrote:

> We're trying to use ActiveMQ as the queue for a distributed set of
> calculation servers (think many machines with 64-cores on-board).
>
> Imagine a single queue with N consumers.  The only efficient way to scale
> is
> to use consumer prefetch.  However.  Each of our messages can take between
> 3ms and minutes to process (there is a huge range).  When testing with 3ms
> per message (with prefetch of 0 or 1) it is not scaling linearly past ~32
> consumers.
>
> Is there anyway (either through configuration, programming, etc.) to make
> the prefetch a variable/dynamic value?  Ideally, I'd like to add hints to
> the message.
>
> Any suggestions would be much appreciated!
>
> Sean
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Variable-dynamic-prefetch-size-tp4658760.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 

**

*Hiram Chirino*

*Engineering | Red Hat, Inc.*

*hchirino@redhat.com <hchirino@redhat.com> | fusesource.com | redhat.com*

*skype: hiramchirino | twitter: @hiramchirino<http://twitter.com/hiramchirino>
*

*blog: Hiram Chirino's Bit Mojo <http://hiramchirino.com/blog/>*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message