Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-users@geronimo.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Message-ID: <45468744.5070600@exist.com>
Date: Tue, 31 Oct 2006 07:14:12 +0800
From: Adrian Co <aco@exist.com>
User-Agent: Thunderbird 1.5.0.7 (Windows/20060909)
MIME-Version: 1.0
To: activemq-users@geronimo.apache.org
Subject: Re: Log Aggregator
References: <7081841.post@talk.nabble.com>
In-Reply-To: <7081841.post@talk.nabble.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Interesting.

Kevin Kessler wrote:
> We are trying to build a system that could be thought of as a sort of log
> aggregator, with a small handful of listeners, and a much larger number of
> publishers (~150).  The messages are going to be 2-5K in size, and we may
> burst up to 3000 messages per second.  My main concern is that no Subscriber
> or JMS issue affects the publishers.  That means no flow-control, no-broker
> problem causing a publisher to hang, etc.  If something happens I just want
> to lose messages, and the publishers continue their real jobs unimpeded.  
>
>   
Publish to a topic with no durable subscriber and use non-persistent 
delivery and you should be fine I think. Though you might run into the 
slow consumer problem, you can use the eviction strategy of ActiveMQ to 
handle it.
http://www.activemq.org/site/slow-consumer-handling.html

> Most of the publishers will be JBoss, although some are WebLogic.  I’ve
> played around with the embedded broker in JBoss, using log4j’s JMS appender
> to publisher messages, and I was very happy with the performance.  I think I
> don’t want to use a traditional hub and spoke sort of configuration, where
> the JMS clients connect to a centralized cluster of brokers, because I don’t
> want to make a remote network call in-line with the publishing system’s
> actual function call just for logging, and I don’t need durable
> subscriptions or delivery guarantees that the hub and spoke design lends
> itself to.  I figured a more peer-to-peer approach of embedded brokers would
> match the requirements better.  My initial idea was to use unreliable
> multicast to transmit the messages between the brokers, but, reading your
> web site, it doesn’t look like that is a design that you favor for a
> production environment.  I’m concerned that making a TCP connection to every
> other broker in the environment is going to be a big problem, especially
> since the vast majority of the brokers are publishers, and have no interest
> in what the other brokers are doing.
>   
I would suggest using an embedded broker for each application that is 
doing the logging, this decouples your publisher from any external 
broker. Then creating a separate embedded broker with your consumer that 
would network with the publishers broker to receive messages. There 
could be a better way, but thats my two cents for now. :)
> Do you think ActiveMQ is a match for this application, and what transport
> protocol would you suggest?
>   
I think ActiveMQ can handle it. VM transport with a TCP transport for 
the publisher and consumer I think.
> Thanks,
> Kevin Kessler
>
>
>
>