qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GS.Chandra N" <gs.chandra...@gmail.com>
Subject Re: Memory pile up on broker
Date Mon, 09 Mar 2009 06:16:32 GMT
Adding some more details -

All messages are sent to a new exchange created thus "qpid-config add
exchange headers Feeds". The exchange is shown as not durable in qpid-tool.

There is no increase in size of memory unless subscriptions are created.

When there are no subscriptions and messages are being pumped in ,

                         msgRecieves is equal to msgDrops
                         byteRecieves is equal to byteDrops
                        msgROutes and byteRoutes are zero

                         All other stats for the exchange shows zero

 Though i have 6 server processes pumping in messages and one qpid-tool I
see only 2 queues.They are named mgmt-HOST.port and repl-HOST.port. I
suppose these are created by qpid-tool and not by the exchange processes
since they do not create any queues for sending messages.

When there are subscriptions created and messages are being pumped in,

                       msgRecieves start to climb and msgDrops stay same
                       byteRecieves start to climb and byteDrops stay same
                       msgRoutes and byteRoutes are non zero

It is clear that the messages are for some reason being cached though its
not clear why.

Thanks in advance
gs

On Sun, Mar 8, 2009 at 11:12 AM, GS.Chandra N <gs.chandran.n@gmail.com>wrote:

> Hi,
>
> I'm stress testing the broker for evaluating the subscription routing
> performance and i see that the memory on the server goes up really high. (1
> gig RSS, almost same Virtual size).
>
> However i'm not able to attribute the reason behind this pile up's. I
> cannot see the pile up attributed to any queues or connections at the
> server.
>
> I do not create any queues from the server process to send messages and
> simply call the python api session.message_transfer to send my messages at a
> rate of 3000 odd mesages per sec(500 bytes payload).
>
> How do i figure out what is happening.
>
> This is my test setups
>
> 1. A 2 CPU XEON server running the broker
> 2. Another server running MULTIPLE python scripts (same script / copies of
> each other) that publish messages to the server
> 3. Another server that runs a few clients that creates a total of 20,000
> odd subscriptions at the broker. Only 3000 subscriptions amongst these are
> unique rest are copies
>
> The client subscriptions never match the messages sent in (to avoid
> bandwidth choking) and therefore the queues created show 0 msges enqued.
>
> Thanks
> gs
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message