qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Java Broker with DerbyMessageStore running out of memory with 20K Queues
Date Wed, 12 Oct 2011 16:22:29 GMT
What version of the client/broker were you using in your test? Can you
send a copy of the code you used to reproduce the issue (it will
probably get scraped by the mailing list if you attach it, so just
paste it in).

Regards,
Robbie

On 12 October 2011 06:21, Praveen M <lefthandmagic@gmail.com> wrote:
> Thanks for your email. I have earlier run the benchmark with
> MemoryMessageStore.
>
> I was able to go up to 50K queues with the exact same test (a message per
> queue), and it took up merely 2.1 GB.
>
> So this seems to be something when I switch to DerbyMessageStore.
>
> Maybe I am missing some setting?
> Or is Derby supposed to perform this way?
>
> Thanks for your help.
>
> Thanks,
> Praveen
>
> On Tue, Oct 11, 2011 at 8:44 PM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
>> Hi Praveen,
>>
>> Do you notice the same behavior even when you run the broker without Derby
>> message store?. AFAIK this has nothing to do with the persistence storage
>> you use.
>>
>> Thanks,
>> Danushka
>>
>> On Wed, Oct 12, 2011 at 4:14 AM, Praveen M <lefthandmagic@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I'm an apache qpid newbie and am trying to benchmark Qpid Java Broker to
>> > see
>> > if it could be use for one of my usecase.
>> >
>> > My UseCase requires the ability to create atleast 20K persistent queues
>> and
>> > have them all running in parallel.
>> >
>> > I am using the DerbyMessageStore as I understand that the default
>> > MemoryMessageQueue is not persistant across broker restarts.
>> >
>> > I'm running the broker with a heap of 4GB and options QPID_OPTS set to
>> > -Damqj.read_write_pool_size=32 -Dmax_prefetch=1
>> >
>> >
>> > My test does the following:
>> >
>> > 1) Creates a queue and registers a listener on that queue. I do this upto
>> > 20K times for 20K distinct queues. I create the queues with the following
>> > option
>> >    create: always , node : {type : queue, durable : true}}
>> >    - this step goes quite fine.  I was monitoring the memory usage during
>> > this step and it almost always stayed stable around 500-800MB
>> > 2) I produce messages for the queues (one message for each queues) and
>> the
>> > messages are consumed by the registered handlers in step 1.
>> >    - When this step starts, the memory usage just shoots up and exhausts
>> my
>> > 4GB memory all together.
>> >
>> >
>> > Can someone please help me explaining why I am seeing this kind of a
>> > behavior?
>> >
>> > Also, Can you please point out if I'm missing some setting or doing
>> > something completely wrong/stupid?
>> >
>> >
>> > Thanks,
>> > --
>> > -Praveen
>> >
>>
>
>
>
> --
> -Praveen
>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message