qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: Reduce Sender traffic
Date Fri, 27 Apr 2012 08:47:16 GMT
I suppose as this point I'd start asking some philosophical questions.....

In general pub/sub is a fire and forget model and is most commonly used 
when you want to decouple producers and consumers, what you are talking 
about is essentially introducing a level of coupling between producers 
and consumers.

Gordon has identified a good solution and the binding information can of 
course be used to steer your producer. As has been stated this has the 
clear advantage that your architecture (well your physical architecture) 
doesn't need to change much.

I say physical because I'd assert that it is a change to your logical 
architecture and this is a qpid specific approach not a "general" 
messaging pattern. Not that that's necessarily a bad thing, qpid is 
great and I'd do similar things myself, I'm just suggesting that you go 
down this path with your eyes open.

An alternative approach might be to take the view that if only about 1% 
of consumers are interested in the messages then a request/response 
architecture might be a better fit.


I don't know your problem space and I'd suspect that Gordon's suggestion 
is probably a quicker and simpler solution, and if you don't control the 
consumers your options are limited etc.

The QMF2 API has an interesting approach it uses what might best be 
described as "query subscriptions" where a QMF Console requests 
management objects specified by a "query", the QMF Agent in this context 
acts as a data producer and pushes back only messages meeting a 
particular "pattern" to the consumer.

All of this stuff is way more complex than Gordon's suggestion, but 
sometimes taking a step back and re-evaluating the overall problem space 
is useful, just in case - at least then you'll be confident that any 
implementation decision that you take will be purely an implementation 
choice.

Frase

On 26/04/12 10:19, Bruno Matos wrote:
> On Wed, 2012-04-18 at 15:41 +0100, Gordon Sim wrote:
>> On 04/18/2012 11:19 AM, Bruno Matos wrote:
>>> Hi,
>>>
>>> I have a sender that publishes about 10MB of messages per second for
>>> different topics. Only about 1% of that traffic is delivered because
>>> only about 1% of the topics are bound at a time. Is it possible, or
>>> would it be desirable, that the sender only effectively send the
>>> messages for witch there is a bound queue?
>> I can see a couple of approaches that could work to limit the sending of
>> unwanted data.
>>
>> One is to subscribe to QMF event messages to detect binding and
>> unbinding at the sender, and deduce from that whether it is worth
>> sending anything.
>>
>> The other is to have the receivers explicitly send control messages
>> themselves indicating when they want and when they no longer need
>> particular data sets.
>>
>> Of course a lot depends on the nature of the data.
> Thank you Gordon, sorry for the late reply.
>
> I think the first option may solve the problem with little changes in
> architecture. I still have to measure the impact of the events'
> subscription.
>
> Thank you.
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message