qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wisler, Trina" <Trina.Wis...@hp.com>
Subject RE: ring queues
Date Tue, 01 Nov 2011 22:27:23 GMT
Thanks for your reply (yes it was a while ago).  We finally upgraded to qpid .12 and we are
seeing the same behavior.  

My test setup:
** C++ brokers, C++ clients
** Providers that publish simple messages
** Client will not consume any messages until I hit the "go" button essentially as I am wanting
the queue to fill up and overwrite itself for a while.
  
If I have a queue size of 10 (for simplicity), and I back up the queue to where it starts
overwriting (I write about 1000 messages), and then let my client start consuming once all
messages have been published - it should only read 10 messages.  But instead, it always reads
between 380 and 397.  I find this highly odd.  Here is what qpid-tool shows for a few of the
queue attributes.

    arguments              {u'qpid.max_count': 10, u'qpid.policy_type': u'ring'}
    msgDepth               10
 
I am using the C++ API as show below. 

void qpid::client::QueueOptions::setSizePolicy	
  (	QueueSizePolicy sp,
	uint64_t maxSize,
	uint32_t maxCount 
  )		
 
The documented qpid example is as such:
 _______________
Create a queue that will support 1000 messages into a RING buffer
#include "qpid/client/QueueOptions.h"
 
    QueueOptions qo;
    qo.setSizePolicy(RING,0,1000);
 
    session.queueDeclare(arg::queue=queue, arg::arguments=qo);
 ________________
 
My sample client snippet of code is this (modeling mostly the above example):

   qpid::client::QueueOptions qo;
   qo.setSizePolicy(qpid::client::RING,0,10);
   session->queueDeclare(client::arg::queue=queue, client::arg::exclusive=false,
                         client::arg::autoDelete=true ,client::arg::arguments=qo);
 
 
So, back to my example, why do I receive more than 10 messages?
 
I did another quick test to see what would happen if I increased my queue size.  I created
a queue size of 100.  I produced 54875 messages, backed up the queue and didn't allow my client
to consume any.  Once I was done producing message, I allowed my client to consume and my
client received 2384 messages immediately.  
 
This is what qpid-tool showed at the moment before I finally allowed my client to consume
(you can see there is only 100 messages on the queue):
 
    arguments              {u'qpid.max_count': 100, u'qpid.policy_type': u'ring'}
   msgTotalEnqueues       54875
    msgTotalDequeues       54775  -->these dequeues are due to the ring queue overwrites
    msgDepth               100

I do not understand this behavior.  

All we want is to have a ring queue of size X and we expect that if we hit X then older messages
will be overwritten. Period.   

Thanks in advance for your time,
Trina Wisler

-----Original Message-----
From: Carl Trieloff [mailto:cctrieloff@redhat.com] 
Sent: Wednesday, October 05, 2011 9:29 AM
To: users@qpid.apache.org
Subject: Re: ring queues

On 10/05/2011 11:07 AM, Wisler, Trina wrote:
> Hi,
>
> I have a queue setup with max message count of 100 and as a ring queue.  The single client
creating that queue will consume 1 message and then wait indefinitely (for the purpose of
my test).  I am producing a total of 250 messages.  As I would expect, qpid-tool reports that
250 messages have been enqueued and 150 have been dequeued (presumably due to the oldest messages
being overwritten).  Here is a snippet from qpid-tool
>
>     property   arguments              {u'qpid.max_count': 100, u'qpid.policy_type': u'ring'}
>     property   altExchange            <NULL>
>     statistic  msgTotalEnqueues       250 messages
>     statistic  msgTotalDequeues       150
>    statistic  msgTxnEnqueues         0
>     statistic  msgTxnDequeues         0
>     statistic  msgPersistEnqueues     0
>     statistic  msgPersistDequeues     0
>     statistic  msgDepth               100
>
> However, when I resume my client, I would expect that it only will receive the 100 remaining
messages on the queue, but instead it completes with all 250.  I'm slightly confused by this
because the definition of a ring queue states : ""RING : start overwriting messages in a ring
based on sizing. If head meets tail, advance head".  The behavior I want is for it to overwrite
older messages once the queue reaches max size.
>
> We are using qpid .06, working on upgrading to .12.
>
> Thanks in advance.
> Trina Wisler
>

The behaviour you see seems incorrect. The ring queue implementation has
been refactored between 0.6 and 0.12 so if you are going to update, if
you can do that and report back, this make is easier to try work out
what is going on if we are both on the latest version.

btw, which client are you using, not that it should matter.

Carl.




---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message