Mailing-List: contact users-help@qpid.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@qpid.apache.org
Received-SPF: pass (athena.apache.org: domain of gs.chandran.n@gmail.com
 designates 209.85.217.174 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=YBG/laaGq+S4iqu6BIvhWu/wYSpv0w/Cc6+fmLUYE9f6T9qOQq1xgjaBbS8VzSGDrz
         dan/OrBPVtoiffNL2kGD29nixHNHG9AstHEclN9TPSKDyBIpL3PIwf+iynQk87HFi93G
         nmkQhfko4K0QJNBauM2ggQbssOMOaJqo0tC2U=
MIME-Version: 1.0
In-Reply-To: <1802abc00903120241t42d035cei41722da1e9647282@mail.gmail.com>
References: <1802abc00903072142p2c5de8a9i5a7048c6f1d9c537@mail.gmail.com>
	 <1802abc00903090937ja6ff557y4e945927c97f4318@mail.gmail.com>
	 <49B5751E.2060701@redhat.com>
	 <1802abc00903092039n41024756x3648e308a58f8ef0@mail.gmail.com>
	 <49B62A56.1090907@redhat.com>
	 <1802abc00903100240g583782dayc20ff95f4abf01a@mail.gmail.com>
	 <49B63BF4.10407@redhat.com>
	 <1802abc00903102334t563f0686w61dff76348853368@mail.gmail.com>
	 <49B818AA.4030306@redhat.com>
	 <1802abc00903120241t42d035cei41722da1e9647282@mail.gmail.com>
Date: Thu, 12 Mar 2009 15:45:23 +0530
Message-ID: <1802abc00903120315p2a940cf8q139780b8cdc626d0@mail.gmail.com>
Subject: Re: Memory pile up on broker
From: "GS.Chandra N" <gs.chandran.n@gmail.com>
To: users@qpid.apache.org
Content-Type: multipart/alternative; boundary=001636163cc72bf74e0464e9437f

--001636163cc72bf74e0464e9437f
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi,

I found that if i kept the qpid-tool open long enough all my clients and
publishers would throw exceptions and come out.

Earlier when i was performing my tests i had 3 putty windows opened to all
the 3 servers and it seems as if, i was keeping qpid-tool opened just long
enough to cause build up but closing it to run top and so on a so forth such
that the memory piled up but did not cause timeouts at the client either.

Observations

1.When there are subscriptions and a qpid-tool around messages seem to pile
up. Why is this so? *
*
2. When there are no subscribers and no qpid-tool the publishing processes
cause the CPu to go to 90% at the publishing box. But when i started up
subscribers, the publishing box CPU fell to 0% and all the python processes
piled up memory.

Concern : I'm not able to find out here if the publishers are still able to
send out messages at the speed that it should when subscriptions are around.
Or perhaps the broker is not able to pull enough messages - iam not sure
which of this is happeneing. Probably latter? How do i tell?

If i use qpid-tool to connect to the broker at this point, every time i hit
show exchange i get the same stats with the same time stamp. it looks like
the broker is too busy trying to match subscriptions even to refresh the
stats.

3.Concern : Once the subscriptions are made, the CPU at the broker box* *(high
end dual core XEON with ht) goes to 90%.  Even at this level, i'm not sure
the broker is able to match and discard all the messages fast enough (due to
2nd observation). Or how do i tell?


Thanks
gs

ps : Is there a sure shot way to find out the message rates ? (I currently
use qpid-tool show exchange command to find the no of msgRecieves and divide
by time-dfference from 2 times the command is run)

On Thu, Mar 12, 2009 at 3:11 PM, GS.Chandra N <gs.chandran.n@gmail.com>wrote:

> I'm able to reproduce the issue, when i keep qpid-tool open all the time
> with the mgmt switches.
>
> However i'm not able to do so if qpid-tool is not open (did not try with
> qpid-tool and no mgmt switches).
>
> Thanks
> gs
>
>
> On Thu, Mar 12, 2009 at 1:31 AM, Gordon Sim <gsim@redhat.com> wrote:
>
>> GS.Chandra N wrote:
>>
>>> ps : Please find attached the scripts that create the issue
>>>
>>
>> I'm afraid I wasn't able to recreate the issue with your tests. Memory
>> stayed reasonable and constant as the test was running (5 servers, 5
>> clients).
>>
>> I'm baffled at this point as to what it is you are seeing.
>>
>>  broker - runs on a dedicated box
>>> qpidd -d --port=5672  --mgmt-enable=yes  --mgmt-pub-interval=1 --auth=no
>>> --log-source=yes --log-to-file=/tmp/somename
>>>
>>
>> Do you have qpid-tool or anything else running during the test? Does
>> turning management off have any impact on the issue?
>>
>>
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>

--001636163cc72bf74e0464e9437f--