activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shimik <shim...@gmail.com>
Subject Re: Best practice for using Topic in medium to large cluster
Date Tue, 22 Jul 2008 07:42:51 GMT

Although it is not the same product and not the same scale (hopefully it will
get to that :-)). Think of a system like youtube where data is big and both
the data and traffic keeps on growing. As opposed to youtube (I guess) in
this case most of the messages are big 1-10M (can be even more but I expect
it to be <5M most of the time) and a good percentage of them needs to get to
all of the consumers.

In the beginning there will be 2 producers (for redundancy) and 10
consumers. The system needs to be scalable. I am expecting both traffic and
data to grow rapidly. The number of producers will grow with traffic growth
and the number of consumers will grow with traffic and data growth.  


James.Strachan wrote:
> 
> 2008/7/15 Shimik <shimi.k@gmail.com>:
>>
>> What is the best practice for using topic in medium to large cluster?
>>
>> In the topologies document (http://activemq.apache.org/topologies.html)
>> it
>> says:
>> "Multicast is great in development though often you might want to disable
>> this feature in production and have well known servers fixed on specific
>> machines. Often socket based communication (using pointcast) is much
>> faster
>> & better for heavy-lifting - particularly on Java - so we tend to
>> recommend
>> to use multicast mostly for discovery and use TCP / SSL for your heavy
>> duty
>> messaging."
>>
>> In a system where all the clients need to receive all the messages and
>> some
>> of the messages can be really big (a few megabytes and maybe more)
>> 1. shouldn't multicast be used? What about JGroups or JRMS?
> 
> We recommend TCP on ActiveMQ as (i) we can't use JGroups due to its
> license and we don't yet have a solid reliable multicast
> implementation.
> 
>> 2. Wouldn't it choke the network if I send all the messages to all the
>> clients in TCP?
> 
> I doubt it; these days you're usually more CPU bound than network
> bound if you've a reasonably good network
> 
> 
>> 3. Wouldn't it choke the network having well known servers fixed on
>> specific
>> machines?
> 
> Not necessarily
> 
> 
>> 4. Does anyone have any experience with this use case?
> 
> Could you give us an idea for how many producers/consumers and what
> kinda message throughput rates you're thinking of?
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Best-practice-for-using-Topic-in-medium-to-large-cluster-tp18460043p18584133.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message