qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: Messaging scenarios
Date Tue, 03 Jan 2012 11:28:34 GMT
On 12/29/2011 07:00 AM, Kalle Kärkkäinen wrote:
> Hi,
>
> I'm developing a system using QPID. I'm considering the options for
> different queues, and would like some opinions and help with the issues
> I have ran in to.
>
> At the moment I'm using the C++ broker with Java clients.
>
> 1) Configuration queue
>
> I have a configuration message, and I need a queue to store it. Last
> Value Queue seems to be the ticket on this, especially if it would be
> durable. The queue get set with qpid-config with parameters
>
> --order=lvq --durable --limit-policy=ring --max-queue-count=1
>
> and it seems that after QPID restart, there are no messages there. Am I
> missing something?

As Bruno pointed out, you need to ensure the messages are also marked 
durable.

Also, in passing, it seems redundant to have a ring queue of 1 message 
and set the LVQ option. If there is only ever one message on the queue 
the LVQ matching will never be needed (each message will simply replace 
the existing one, regardless of the key).

If you do want to use LVQ I would also recommend using 
qpid.last_value_queue_key to set the property to key on. That will also 
configure more intuitive behaviour in place of the old (now legacy) LVQ 
implementation. E.g. the qpid-config parameters might be: --argument 
qpid.last_value_queue_key=my-property.

> 2) one-to-many messaging
>
> Earlier (years back, m4 time) I asked the same thing, and I suspect the
> same answer: The way to achieve one-to-many messaging is pubsub setup
> with private temporary queues with routing rules to move messages from
> pub-queue to said sub-queues. Is this still the case?

There is no pub-queue, there is (in AMQP 0-10) an 'exchange' to which 
the temporary queues are all bound. Publishers send message to this 
exchange and all queues with a matching binding receive a copy of it.

If you are using JMS, this corresponds to a Topic type Destination. 
There are various ways to configure that, but I think the simplest is to 
create an exchange for each logical topic and simply use the exchange 
name to define/create the Destination. That way everything is taken care 
of by the JMS library.

> The real world case here is that we are syncing servers with messages,
> sending them to federated QPIDs and processing the data only after it
> has been through the federation (passed to all QPID nodes). Is this the
> way to go? Corosync setup would be difficult because of the broadcast
> requirement, QPID nodes are not in same network.
>
> 3) many-to-many messaging
>
> There are times when we have multiple pubs in the pubsub scheme. How do
> we do this? Would it be more routing inside the QPID, or what?

Really this boils down to the same case as one-to-many. You have a Topic 
and you can have one or more MessageProducers (in JMS) created for it.

The one special aspect you may want to consider is total ordering. I.e. 
do you need all subscribers to see the messages in the exact same order?

In the case of a single producer this will always be the case as the 
messages are causally ordered anyway. However with multiple concurrent 
producers the interleaving of messages from different producers may be 
different on different queues. To work around this you can make use of 
an extended option on the c++ broker that synchronises all publications 
to ensure each subscriber sees the exact same sequence. This of course 
does imply some penalty in throughput due to reduced concurrency.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message