qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: Is it possible to have persistent circular queues
Date Mon, 14 Mar 2011 11:34:50 GMT
On 03/13/2011 06:29 PM, fadams wrote:
> Hi Again
> Re the response:
> Gordon Sim wrote:
>> When using qpid-config it would be:
>>     qpid-config add queue my-durable-ring-queue --limit-policy ring
>> --max-queue-count N --durable
> This still doesn't seem to be doing what I'd hoped.
> I created a queue as follows:
> qpid-config add queue myqueue --durable --max-queue-count 50000 --file-size
> 5000 --file-count 16 --limit-policy ring
> so this should have created a queue of size 5000*64K*16 ~5GB
> My producer is sending messages of 50K so with max-queue-count of 50000 I'd
> expect 50K * 50K to be used ~2.5GB
> However when I run my producer with no consumer it happily runs through with
> no exception (I had a produce run of 100000 items to see what happens) but
> here's the rub - when I fire up my consumer it only receives 2096 items not
> 50000. 2096*50K ~104800000 which is roughly the default-queue-limit
> What I really wanted to achieve is a queue larger than my available memory,
> but I'd also like to silently dump the oldest stuff without throwing an
> exception back to the producer

That you can't do at present. You can either have the flow to disk 
policy that will release message content from memory once the limit is 
reached or you can have a ring policy that deletes the oldest message(s) 
at that point.

> - which leads back to my original
> question......
> So is it possible to set up a very large durable queue that behaves as a
> ring buffer (of roughly the size of the queue as opposed to the
> default-queue-limit size)

This part is possible. However the size of the ring buffer is configured 
separately from the size of the journal. The simplest way around your 
issue above is to set the default-queue-limit to 0. The queue will then 
always have 50000 messages on it, whatever the size of those messages.

There are a couple of bugs I've just noticed and raised JIRAs for that 
prevent either setting the specific queues size to be unlimited[1] via 
qpid-config, or specifying a large enough size in your case[2].

[1] https://issues.apache.org/jira/browse/QPID-3141
[2] https://issues.apache.org/jira/browse/QPID-3140

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

View raw message