qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Ross <tr...@redhat.com>
Subject Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid Dispatch Router component
Date Wed, 09 Oct 2013 19:04:45 GMT
Frase,

This is an excellent post, and I believe quite relevant.  I'll try to 
address your questions at an abstract level rather than point-by-point.  
Your confusion is not unique, but quite justified.

AMQP 1.0 is simply a wire-level protocol specification for symmetric 
point-to-point data communication.  It has very advanced semantic 
capabilities that are inspired by middleware messaging.  AMQP is, in my 
opinion, a game changer for distributed computing.  It opens up 
possibilities that go way beyond client/broker messaging.  In fact, it 
holds the potential to revolutionize the much broader world of data 
networking and distributed systems because it offers capabilities that 
TCP/IP and HTTP can't touch.

Apache Qpid is the premier open source implementation of AMQP.  In the 
AMQP 0-{8,9,10} era, Qpid was pretty much just another 
middleware-oriented-messaging (MOM) system based on the emerging 
standard.  With the ratification of AMQP 1.0, which is significantly 
different (and superior) to the predecessor specifications, Qpid is 
undergoing a bit of a transition.

We have, of course, updated our brokers and clients to use the new 
specification and thus continue to serve the MOM use cases.

The Proton project was created for the purpose of promoting AMQP and 
making it easy to integrate into many different environments (I believe 
we've lost sight of that purpose somewhat).

The Dispatch project was created for the purpose of exploring how AMQP 
can be applied in non-broker networks.  Dispatch is a form of advanced 
interconnect based on AMQP with the goal of building large-scale and 
high-performance networks that take advantage of the unique capabilities 
of the protocol.  It is intended to extend the reach and usefulness of 
brokers.

There are a lot of new ideas floating around, some of them overlapping.  
I think Qpid is a perfect place to explore and develop new technologies 
based on AMQP.  This will cause some confusion and force us to work at 
articulating what we are doing and thinking, and it will be people like 
you who will prompt the important discussions.

Regards,

-Ted


On 10/09/2013 02:22 PM, Fraser Adams wrote:
> Hey all,
> The thread below on the dev list has prompted me to ask something that 
> I've tentatively mentioned before, but am still a bit embarrassed to 
> raise 'cause it probably makes me seem a bit stupid :-( here goes 
> anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to 
> lack of time, but also partly due to lack of understanding of how it 
> "all hangs together", the new website helps a bit - but TBH I'm still 
> left scratching my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond 
> just the Qpid "product set", but there's a "protocol engine" and a 
> "messenger API" and I'm not even that clear on the relationship 
> between the two of those - for example could one use the protocol 
> engine completely independently (is there an engine API?) or is the 
> messenger API intended to be the lowest "unit of currency", what would 
> be the benefit using the raw engine?
>
> Then beyond that there's the relationship with say qpidd and 
> qpid::messaging. Now I'm aware that when the Proton libraries are 
> detected qpidd and qpid::messaging get built with Proton support, I'm 
> "guessing" that in that case the relationship analogous to that of 
> qpid::client where qpid::client was the low level AMQP speaking API 
> and qpid::messaging provides a higher level abstraction, so I *think* 
> that's the relationship with Proton there - but I'm not sure? Is the 
> proton API close to the AMQP 1.0 specification in say the way that 
> qpid::client was?
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's 
> a more peer-to-peer relationship and indeed the Proton tests seem to 
> have msgr-recv and msgr-send talking directly to each other without a 
> broker. So that leads me to ask the question what's the relationship 
> with the broker - in other words what services are provided in 
> messenger, what are enhanced in qpid::messaging and what are layered 
> on top of that via the broker (and how does the addressing and routing 
> work?).
>
> Some examples of where I'm befuddled include how does subscription 
> work at a peer to peer level? For example I think that exchange nodes 
> are only something I've heard discussed in the context of qpidd and 
> similarly I think the same is true of message selectors, so does 
> Proton only provide low level network connectivity and data 
> serialisation (and possibly single client queue) and all the other 
> stuff needed for connecting a network of clients are part of the 
> broker services.
>
> I suppose what I'm really asking is what "services" are provided at 
> each "layer" of the Qpid "stack" - clearly you can do useful stuff 
> with just Proton - but what stuff and what are the limits? What would 
> you then get from qpid:messaging and what then does the broker throw 
> into the mix. Are there any diagrams that illustrate this sort of 
> relationship?
>
> The dispatch router adds yet more nuance into the mix. From my 
> (limited) understanding it seems to offer at least some of the same 
> services as the broker - but I'm not quite sure what. In my case I've 
> got a very large federated topology and I have lots of left hand 
> systems feeding in to fewer systems towards the right. Given that it's 
> only on the right hand side broker that I have lots of consumers doing 
> complex subscriptions and the rest of the brokers are employing fairly 
> simple queue routes I'm thinking that the dispatch router could 
> ultimately be something to "tidy up" the left hand side of my system - 
> but I'm not quite sure.
>
> Apologies if these seem silly questions, I'm sure that the answers are 
> obvious to those who've been involved at the architectural stages, but 
> ultimately from my perspective the overall holistic architecture isn't 
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the 
> programming book 
> http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html 
> that seems to mention even how to connect via AMQP 1.0 vice 0.10. I 
> think that it has been mentioned on the mailing list by Gordon so I'm 
> sure I could dig the info out, but is it missing from the docs (or am 
> I just not looking hard enough). On a similar note for Proton the 
> msgr-send and msgr-recv examples are fine as far as it goes, but I'm 
> thinking that to figure out how to do anything more complex my best 
> bet is likely to be to "reverse engineer" the qpid::messaging bindings 
> - I can't see anything obvious for how to send a map for example. I'm 
> guessing that Proton is just as erm "nuanced" as qpid::client, so 
> really powerful and flexible, but you have to know what you're doing 
> to get the best (say performance) out of it, the API documentation 
> looks pretty decent to be fair but I'm not sure that's enough to help 
> me drive it really effectively.
>
> On top of that there seems to be a growing number of JMS clients, 
> there's the original AMQP 0.10, there's an AMQP 1.0 one in the main 
> Qpid tree and there's a separate Proton based AMQP 1.0 one that's a 
> separate component (in a similar vein to Proton). I can see that the 
> increased modularisation is a good thing and I assume that at some 
> point the original AMQP 1.0 JMS client will be deprecated in favour of 
> the Proton based one, but at the moment it's all a bit confusing 
> without anything that describes the relationship between then. I'm 
> gleaning what little knowledge I have out of a range of threads on the 
> mailing list and I've probably missed something.
>
> I'm sorry if this comes across in any way as critical in email form, 
> it's really not intended to, I'm just keen to finally make a proper 
> start on my AMQP 1.0 journey and to be honest I feel a little out of 
> my depth at the moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
>> Hi Ted,
>>
>> I think before we make this a full sub project, it would be good to have
>> clarity on exactly the proposed scope of Dispatch, how it is expected to
>> interact with other components within Qpid, or within wider AMQP 
>> networks.
>> I think in retrospect we didn't do this clearly enough with Proton (for
>> example).
>>
>> Moreover I would personally like to understand which AMQP standards 
>> it will
>> be looking to implement, and which not.  For instance I notice this 
>> line in
>> the docs for Dispatch:
>>
>> *Address**Description* /_local/agentThe management agent on the attached
>> router/container. This address would be used by an endpoint that is a
>> management client/console/tool wishing to access management data from 
>> the
>> attached container.
>> Which doesn't seem to conform with the proposed management specification
>> for AMQP, nor does the document make any mention of how dispatch is 
>> to be
>> managed.
>>
>>
>> Cheers,
>> Rob
>>
>>
>> On 9 October 2013 17:22, Ted Ross <tross@redhat.com> wrote:
>>
>>> The AMQP Router project (Qpid Dispatch, announced previously on the 
>>> user
>>> list) is gaining in community interest and is nearing the point where a
>>> first release is appropriate. In preparation for a release, I 
>>> proposethat
>>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>>> projects. This involves:
>>>
>>> 1. Moving the code from qpid/extras to
>>> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>>> ,
>>> 2. Requesting, by vote, the creation of a JIRA project to track its
>>>     issues and releases.
>>>
>>> Unless there are objections, I will move forward with the above two 
>>> tasks.
>>>
>>> Regards,
>>>
>>> -Ted
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message