From users-return-30611-apmail-activemq-users-archive=activemq.apache.org@activemq.apache.org Fri Apr 6 09:24:53 2012 Return-Path: X-Original-To: apmail-activemq-users-archive@www.apache.org Delivered-To: apmail-activemq-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 15BA199EF for ; Fri, 6 Apr 2012 09:24:53 +0000 (UTC) Received: (qmail 21203 invoked by uid 500); 6 Apr 2012 09:24:52 -0000 Delivered-To: apmail-activemq-users-archive@activemq.apache.org Received: (qmail 21129 invoked by uid 500); 6 Apr 2012 09:24:52 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 21117 invoked by uid 99); 6 Apr 2012 09:24:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Apr 2012 09:24:52 +0000 X-ASF-Spam-Status: No, hits=0.6 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gary.tully@gmail.com designates 209.85.216.171 as permitted sender) Received: from [209.85.216.171] (HELO mail-qc0-f171.google.com) (209.85.216.171) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Apr 2012 09:24:47 +0000 Received: by qcsp15 with SMTP id p15so1494883qcs.2 for ; Fri, 06 Apr 2012 02:24:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=kiykR5veRQ4Gby9sjwjbTW08r6iZwqru7bII58K5fuU=; b=MxYoPoXr4G6IdkPUAhJJsfUJi3LDwrOhe4k6slqlegxwhXAVHKppxAvhV11SEsBEN/ 9PYYlTGJw71AUzGz8E1KphV7u/O82rl4G+4iQuPFcte3fBKW+CuYlUvUkp+wNPLp6AjE 7Ku8QQOp/puVSUg76nES4Syhm4LySbQB8py3JPOOGO+9xmNZ5DDEav2+1JkM58jMmN3O Ce+hCGNqjekzULwu/8Bs5NlhGM2dxjYh9O4EvmlacDiuYp/6eR3uohYGTzx33mb4gKy2 XW9BcJVxpMZ734z4Ie2yw03yOLeSYXkuvgRbtIrLonZnZK8C6Ru4Rr5qDAmTPLwwm0iu 2uCg== MIME-Version: 1.0 Received: by 10.224.31.69 with SMTP id x5mr6747559qac.67.1333704266720; Fri, 06 Apr 2012 02:24:26 -0700 (PDT) Received: by 10.229.158.4 with HTTP; Fri, 6 Apr 2012 02:24:26 -0700 (PDT) In-Reply-To: <1333652015677-4535723.post@n4.nabble.com> References: <1333652015677-4535723.post@n4.nabble.com> Date: Fri, 6 Apr 2012 10:24:26 +0100 Message-ID: Subject: Re: Inter Broker overhead is very high From: Gary Tully To: users@activemq.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org The inter broker bridges can be bottle necks as they funnel all messages over a single connection. Often it is beneficial to partition destinations across multiple network bridges using the destination filter or include/exclude attributes In a simple case where application destinations are easily partitioned; eg: topic:app.X, topic:app.Y. Configuring two network bridges and confining destinations of a particular name to each would help distribute the load. eg: " duplex=3D"true" ...> " duplex=3D"true" ...> On 5 April 2012 19:53, Saiky76 wrote: > Hi - we are trying out a POC with Active MQ for extremely high rate of > traffic. Our most important goal is the ability to scale. The topology > involves topics and for the sake of this post, please consider there is o= nly > one consumer per topic. > > In our reference topology, producers (more than one) keep posting message= s > to topics to any broker in the cluster of brokers =A0- using the failover > protocol that is randomized. Similarly the consumers also attempt to cons= ume > messages from any broker in the cluster. =A0Basically, the producers and > consumers are not aware of the topic location on a specific broker. For t= his > to work, I set up network bridge (duplex) between the brokers. In tests t= hat > had 3 brokers, I ensured all the brokers had a direct network bridge to a= ll > other brokers - to ensure there is only one broker hop for the message to= be > forwarded to the broker on which the topic consumer is listening. > > The number of connections (both consumers and producers) on a single brok= er > never exceeded 250+. > > The vertical scaling results for a single broker is quite impressive. But > once I try my scaling tests, adding 2 or 3 brokers does not help and the > throughput that I got from one broker is exactly the same for the scaling > tests with 2 or 3 brokers. The producers and consumers are randomly writi= ng > and consuming from the brokers as described above. Actually the load is > balanced almost evenly in these writes and consumes. Please note I ensure= d > enough CPU, memory on the consumer side. > > After doing some analysis, I feel the inter broker communication is > responsible. =A0When I tried with 2 brokers and let us say the total numb= er of > messages consumed is 200k, then the messages exchanged between 2 brokers = is > 100k. Similar is the case when 3 brokers are tested - exactly 66% of > additional messages are exchanges between brokers. =A0Looks like each bro= ker > is doing exactly what is max capable of but the additional power of addin= g > broker is nullified by the network hops. > > Any ideas, any advise would be great? I am wondering whether horizonatal > scalability is actually designed for this. Please note I read the whole o= f > ActiveMq in action book and literature avaialable online on MQ. Please ag= ain > note the additional network hop is always one and that is predictable. > > I am trying this on MQ 5.5.1. Thanks. > > > > -- > View this message in context: http://activemq.2283324.n4.nabble.com/Inter= -Broker-overhead-is-very-high-tp4535723p4535723.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. --=20 http://fusesource.com http://blog.garytully.com