qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Lavigne du Cadet" <j.lavigneduca...@abc-arbitrage.com>
Subject Load balancing messages to consumers that have different performances characteristics
Date Mon, 07 Mar 2011 14:25:04 GMT
Hi,

I have the following scenario : 

I'm pushing 100 messages to a queue shared by 2 consumers. Both
subscribers subscribe to the queue in pre-acquire mode and explicit
mode. After each message is handled, each subscriber accepts the message
to remove it from the queue. The pseudo codes look likes that : 

OnMessageTransfer(message) :

    DoSomethingWithMessage(message)

    Session.MessageAccept(message)

The messages are load balanced correctly, each message is processed once
and only once, but we discovered that it doesn't take into account the
processing time for each consumer. For instance, let's assume that
consumer A is taking 50ms to process a message and consumer B is taking
5seconds. Ideally, consumer B should start processing 1 message and in
the meantime, consumer A should process the 99 others. However, what
happens is that consumer B will actually process 25 messages in 50
seconds while consumer A will process the 75 others in ~4 seconds and
will idle. The client api seems to prefetch the messages, which is
clearly non optimal in this situation.

How can we solve this problem?

We're using Qpid cpp 0.5 and the fully managed c# 0-10 client API, not
the cpp bindings (but my understanding is that this behavior is not
linked to the implementation of the API) 

Message cross posted on stack overflow if someone prefers to answer
there : 

http://stackoverflow.com/questions/5220808/qpid-load-balancing-messages-
to-consumers-that-have-different-performances-char
<http://stackoverflow.com/questions/5220808/qpid-load-balancing-messages
-to-consumers-that-have-different-performances-char> 

Regards,

Julien

 


-------------------------------------------------------------------------------------------
ABC arbitrage, partenaire officiel du skipper Jean-Pierre Dick (bateau Virbac Paprec 3), engagé
avec Loïc Peyron sur la Barcelona World Race 2011, course autour du monde en double sans
escale // ABC arbitrage, official partner of skipper Jean-Pierre Dick (Virbac Paprec 3 boat),
racing with Loïc Peyron in the 2011 edition of the Barcelona World Race (double-handed non-stop
round-the-world race).
P  Please consider your environmental responsibility before printing this email
*********************************************************************************
Ce message peut contenir des informations confidentielles. 
Les idees et opinions presentees dans ce message 
sont celles de son auteur, et ne representent pas necessairement
celles du groupe ABC arbitrage.
Au cas ou il ne vous serait pas destine,
merci d'en aviser l'expediteur immediatement et de le supprimer.

This message may contain confidential information. 
Any views or opinions presented are solely those of its author 
and do not necessarily represent those of ABC arbitrage. 
If you are not the intended recipient,
please notify the sender immediately and delete it.
*********************************************************************************


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message