qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajith Attapattu <rajit...@gmail.com>
Subject Re: Qpid messaging design
Date Tue, 05 Oct 2010 15:42:45 GMT
On Mon, Oct 4, 2010 at 2:12 PM, felixehm <felix.ehm@cern.ch> wrote:
>
> Hi,
>
> We're currently evaluating Qpid as a middleware solution and during this
> process I stumbled upon some questions which I hoped to get clarified with
> some of your help.
>
> For the evaluation I have to implement a typical scenario on our site. 400
> Metrics (numbers) are published every second and need to be retrieved <
> 300ms by max 10 subscribers. It should still be possible to subscribe to
> individual values only.
>
> In JMS words : 400 topics each subscribed by 10 Message listeners.
>

I would start using the new messaging API here.
Read the following doc to get a good idea.
http://qpid.apache.org/books/0.7/Programming-In-Apache-Qpid/html/index.html

> Now, reading the documentation and the examples the way to go is to send the
> values via a topic exchange and set the routing key for each Message to the
> according value identifier:
> for (i=0;..){
>   message.getDeliveryProperties().setRoutingKey(routing_key+i);
>   async(session).messageTransfer(arg::content=message,
> arg::destination="amq.topic");
> }
> (I'm aware that packing them into one message is very obvious and faster.
> But for various reasons this is currently not possible in our environment.)

>
> The subscriber on the other hand will subscribe using the same exchange, but
> for each subscription a different routingKey.
> for (i=0..){
>  session.queueDeclare(arg::queue=queue+i, arg::exclusive=false,
> arg::autoDelete=true);
>  session.exchangeBind(arg::exchange="amq.topic", arg::queue=queue+i,
> arg::bindingKey=metric1);
> }
>
>
>
>
> 1. Is this the only way to realize the described scenario?
> 2. Why do I need an own queue per routing key per client (400 x 10)?
>    I've tried to move the queueDeclare outside the loop to re-use the
> client-specific queue. I'd imagine that the server then dispatches messages
> with the according routing key to this client's queue. But apparently this
> is not allowed by the client library.

Since you have 10 subscribers you could just have 10 queues, instead
of 4000 queues.
Each subscriber can create a queue and then bind it's queue to the 400 topics.

This is based on the assumption that you don't have to give equal
priority for each topic, as topics that are more active will likely
dominate the queues and the less active topics will have it's messages
being processed few and far apart.
We've had cases where customers had to treat each subscription equally
and therefore 4000 queues were needed.
In that case the subscribers would go in a round robin fashion
consuming from each of it's 400 queues.


> It might well be that I've misunderstood some concepts and would therefore
> I'd appreciate any help on this.
>
> Cheers,
> Felix
>
>
> --
> View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Qpid-messaging-design-tp5600245p5600245.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>



-- 
Regards,

Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message