qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wright <atwri...@mac.com>
Subject Re: Java atomic bindings Was: Re: Queue creation/deletion
Date Mon, 02 Feb 2009 19:28:17 GMT
Hi Carl,

It's a little different to scheduling/job submission - but perhaps not  
that different. Any examples from other systems would be most helpful,  
if you wouldn't mind pointing me in their direction it'd be much  
appreciated.

Cheers,
Andrew

On 31 Jan 2009, at 02:07, Carl Trieloff wrote:

>
> Andrew,
>
> In some other work some of I guys I work with have done have created  
> a low-latency scheduling mechanism via AMQP (Qpid) for the Condor  
> Grid. It sounds like you are trying to achieve something equivalent  
> using a different grid engine. If you like I can point you to doc  
> and code for that integration, either to use, hack, or to copy :-)
>
> Carl.
>
>
> Andrew Wright wrote:
>> Actually this was to integrate with Terracotta. TC needs some sort  
>> of load balancer to get locality of reference (for performance) -  
>> qpid would fit the bill here, via hierarchical topics. There'd only  
>> be a single subscriber on a topic at a time. TC clients are  
>> notified as nodes join/leave, so can take over the processing of a  
>> given topic if the primary processor goes away.
>>
>> As you say, Coherence has this sort of thing built in.  
>> Unfortunately it also comes with a fairly hefty Oracle price tag...
>>
>> Andrew
>>
>> On 30 Jan 2009, at 23:14, Robert Greig wrote:
>>
>>> 2009/1/30 Andrew Wright <atwright@mac.com>:
>>>> Apologies, I've confused the options rather. We'd thought about  
>>>> using the
>>>> dynamic federation features in M4 to get effective point-to-point  
>>>> links
>>>> between 'entry' nodes, and 'processing' nodes, in a distributed  
>>>> app. The
>>>> idea was that each app node would subscribe to a subset of  
>>>> incoming messages
>>>> (for load balancing), and the federation would take care of  
>>>> getting the
>>>> messages to the right place. Ideally there'd be no message loss  
>>>> in the case
>>>> of app node failover, ie, we want to maintain the contents of an  
>>>> incoming
>>>> work queue if the destination point of a dynamically federated  
>>>> queue
>>>> changed. I'd thought perhaps the atomic rebinding feature could  
>>>> be useful
>>>> here.
>>>
>>> And you are using Coherence as the grid, with the grid nodes as the
>>> processing nodes? If so, I would be very tempted to use the
>>> InvocableMap feature of Coherence and have the messages injected  
>>> into
>>> the grid using that method. That will guarantee that if a node goes
>>> down as it is processing a message it is automatically handled by  
>>> the
>>> backup node.
>>>
>>> For the injector nodes, I would using a number of nodes with a queue
>>> to round-robin between the nodes.
>>>
>>> I am not sure how the topic helps here - it will go to all  
>>> subscribers
>>> so unless the operation is idempotent you would have to handle which
>>> subscriber is the "master"?
>>>
>>> RG
>>>
>>> ---------------------------------------------------------------------
>>> Apache Qpid - AMQP Messaging Implementation
>>> Project:      http://qpid.apache.org
>>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message