qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: more memory "leak" woes.
Date Fri, 03 Aug 2012 12:35:52 GMT
Thanks for the quick response Gordon!
>
> For a reliable receiver, the broker needs to keep track of all 
> unacknowledged messages. That will include all prefetched messages 
> (depending on the frequency at which acknowledgements are actually 
> sent it may include fetched messages also of course).
We were beginning to wonder if that was what was happening. I guess 
that's borne out by my experiments with the messaging-consumer capacity.

>
> If the receiver is unreliable (no explicit acknowledgement expected) 
> then (at least in recent brokers) the record will no longer reference 
> the message, but simply keep track of the amount of byte credit it 
> consumed in order to move the window correctly.
when you say recent - what version? I've been testing at home with 0.12 
and when I added the link unreliable stuff to my qpid::messaging client 
the consumption seems stable, so I'm guessing 0.12 is OK. Unfortunately 
at work  the brokers are still 0.8 - I'm pushing to uplift, maybe this 
will help my corner....

>
> Yes, you would need to use FlowControl::messageWindow() instead; the 
> window will be moved automatically by the library unless you choose to 
> handle completions manually. By default qpid::client subscriptions use 
> unlimited credit (meaning the prefetch is unbounded).
Hmmm I tried swapping messageCredit() to messageWindow() to no avail - 
see below

void Listener::prepareQueue(string queue, string exchange)
{
     queue += session.getId().getName();
     cout << "Declaring queue: " << queue << endl;

     FieldTable q_fields;
     q_fields.setString("qpid.policy_type", "ring");
     q_fields.setUInt64("qpid.max_size", 100000000/*1000000000*/);

     session.queueDeclare(arg::queue = queue, arg::exclusive = true, 
arg::autoDelete = true, arg::arguments = q_fields);

     FieldTable b_flds;
     b_flds.setString("x-match", "all");
     b_flds.setString("data-format", "test");

     session.exchangeBind(arg::exchange = exchange, arg::queue = queue, 
arg::arguments = b_flds);

     cout << "Subscribing to queue " << queue << endl;
     subscription = subscriptions.subscribe(*this, queue);

subscription.setFlowControl(FlowControl::messageWindow(100)); // Why 
doesn't this work :-( ???
     subscription.setAutoAck(10);
}

I'm really struggling with qpid::client flow control - I tried looking 
through 
https://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/tests/qpid-topic-listener.cpp 
which I found Googling qpid::client flow control, but as I say when I 
did the

SubscriptionSettings settings;
settings.autoAck = 10;
settings.flowControl = FlowControl::messageCredit(100);

listener.subscription =  mgr.subscribe(listener, control, settings);

It just hung at 100 messages - clearly qpid::client doesn't like me! the 
feeling is mutual :-)

>
> (Yes, I agree its not the easiest API to use and has a fair amount of 
> incidental complexity, hence qpid::messaging!)
Your telling me :-)

>
> Indeed, if you are not using the --ack option then there are no 
> acknowledgements required. Though the federation subscriptions do use 
> infinite credit, this should at least prevent holding on to messages. 
> What version of the broker are you using now? (I recall at one time 
> there was an older broker in part of the system?)
So we're definitely not using the --ack option, but the " federation 
subscriptions do use infinite credit" is troubling. As I say we're using 
0.8 brokers.

So if the "unreliable message" behaviour isn't behaving as expected in 
earlier brokers (say 0.8) and we have a slow consumer hanging off the 
other side of a federated link, then are you suggesting that this just 
might be the cause of our woes?

>
> I could probably get you a simple patch if applying that and 
> rebuilding is an option.
>
It might be an option - thanks for the offer! At this stage having a 
good analysis and understanding is probably most important - so if you 
can help me enable flow control (and suggest how I'd try unreliable 
messaging using qpid::client too if possible) in the client-consumer I 
posted that would be really helpful.

If I can get evidence to suggest that the federation "unreliable links" 
behaviour isn't as effective at mitigating this in 0.8 as it is in later 
brokers that would be useful, as I say I keep arguing that we should be 
upgrading to something newer.

Many thanks again for your help.
Frase



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message