activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: Message group bottleneck?
Date Tue, 10 Oct 2006 15:58:56 GMT
On 10/10/06, javaxmlsoapdev <vikasdp@yahoo.com> wrote:
> James.Strachan wrote:
> >
> >
> >>No - but they are assigned to a single consumer. The effect is kinda
> >>like each consumer having its  own queue and all IBM messages being
> >>put on the same consumer queue - but there really only is 1 queue.
> >
> > Bit confused here. Did you mean if I use message groups there will be only
> > ONE consumer

per message group (i..e. per value of JMXSGroupID) yes


> > but effect will be like having a seperate consumer queue for
> > each symbol? I thought with the use of message groups activemq will use
> > consumer from the consumer pool, one per each symbol? For e.g. If I am
> > using message groups and have 15 symbols and consumer pooling size is 25
> > then there will be 15 unique consumer queue handling unique symbol with
> > the effect of message groups? can you please clarify.

Yes


> >>So if you want load balancing of messages for IBM, then don't use
> >>message groups. The entire reason to use message groups is you want to
> >>preserve message order of IBM messages - which means you must only
> >>have 1 single consumer active at any moment in time to process IBM
> >>messages (and so no load balancing for IBM messages).
> >
> >>If you want to load balance across symbols (so that say each consumer
> >>just processes 1 symbol) then just keep booting up consumers - the
> >>more consumers there are the less symbols each consumer has to
> >>process.
> >
> >>The message group key (JMSXGroupID) is your way of specifying how
> >>parallel to make the consumption of the messages - so to further
> >>increase the parallelisation, you could try figure out how you can
> >>further split up, say, IBM messages. e.g. use IBM_Exchange_Date to be
> >>able to parallelise IBM symbols traded on different days on different
> >>exchanges or whatever. But its your call on how much you care about
> >>ordering on how much you want to parallelize
> >
> > Ordering is a primary requirement so can't compromise on that. Can you
> > adivse then what's the best way of load balancing with ordering?

So in general, you can't load balance and maintain order as load
balancing breaks ordering - to preserve order you have to do things
sequentially, which in JMS means using 1 consumer and 1 thread.

So all you can do is separate the things which have to be ordered into
seperate logical ordered groups, then load balance the individual
message grouups. e.g. using the JMSXGroupID value to define one
particular order. e.g IBM or IBM-Book123-Date-LondonStockExchange or
whatever. To increase the load balancing, use a more specific value of
JMSXGroupID to create more distinct message groups - and have more
consumers

-- 

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message