qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wright <atwri...@mac.com>
Subject Re: Java atomic bindings Was: Re: Queue creation/deletion
Date Thu, 05 Feb 2009 23:05:36 GMT

On 4 Feb 2009, at 22:50, Robert Greig wrote:
>
>
>> The big difficulty we have is a hard requirement for in-order message
>> processing. This unfortunately rules out load balancing from a  
>> queue across
>> active-active client pairs. We'd rather just route incoming msgs to a
>> store-and-forward topic, and accept a small processing pause during  
>> app node
>> failover.
>
> Do you have the requirement for total ordering or is it just a partial
> ordering, i.e. you need to process requests for a given order id in
> the right order (i.e. new/amend/cancel for a given order id)?
>

It's a partial ordering, per instrument. Ie, orders on the same  
instrument need to be processed sequentially.

>> Once a message has been read from the topic, it is written into the  
>> TC
>> cache. At that point we can failover processing to a different node  
>> without
>> any message or data loss.
>
> Does TC now support memory replication between the server nodes?
> Previously it required shared storage which meant the performance
> sucked.
>

I can imagine. It now does in-memory replication between a master &  
slave. They still have the shared storage mode but it's basically  
deprecated these days I believe.

It also does notification of client leave/join (to remaining clients).  
The general idea is we use qpid to split messages between TC client  
nodes (for data locality), and TC notifications for HA (another node  
can take over processing of a given stream of orders).

>>
>> Absolutely - and we haven't ruled anything out yet. We're going to  
>> start
>> with the lowest cost option, and if it doesn't work out, then have a
>> rethink.
>
> What are your throughput and latency requirements?
>
> This is an interesting domain since it blurs the distinction between
> messaging and in-memory data grids.

Most definitely. Like most things, 'as much throughput as fast as  
possible' - according to the business at least! qpid seems more than  
capable of handling our throughput requirements. We're currently  
investigating a few application structure scenarios to see which one  
meets our latency/load balancing needs the best.

Cheers,
Andrew


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message