qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Godfrey <rob.j.godf...@gmail.com>
Subject Re: Java broker exchange questions
Date Sun, 10 Mar 2013 09:45:47 GMT
On 10 March 2013 09:26, Fraser Adams <fraser.adams@blueyonder.co.uk> wrote:
> On 09/03/13 16:46, Rob Godfrey wrote:
>> Yeah... The biggest gap is probable going to be broker <-> broker
>> federation links.  That code did exist previously in the Java Broker,
>> but it disappeared when we ripped out the QMF1 implementation.
> I'm curious here, are you saying that federation existed in the Java Broker?
> As far as I was aware federation was only possible between C++ brokers and I
> thought that it was necessary to do some sort of client side bridge to link
> between C++ and Java Brokers or to connect Java Brokers. I was fairly sure
> I'd seen postings to that effect on the user list. Though as I say I'm
> vastly unfamiliar with the Java Broker and this work I'm doing is my first
> journey there.

Yeah... there was an implementation of the federation code which
worked with the QMF1 implementation.  I'm not sure it was 100%
complete but it covered most of the functionality I was aware of.
Unfortunately the QMF1 stuff kept getting broken and there was no
other way of configuring it (I originally only implemented QMF1 so
that I could add federation to the Java Broker and have a single set
of management tools across the two brokers... it just ended up being
more trouble than it was worth).  I can't remember when we ripped out
the QMF1 implementation (0.18 perhaps, but it wasn't *too* long ago).

> Looking at the 0.20 code I've got installed there is a federation directory
> in
> <qpid-root>/java/broker/src/main/java/org/apache/qpid/server
> but it's empty in 0.20.
>>   We
>> could probably add it back in but it would have to fit within the new
>> internal model for configuration that Alex/Robbie have been working
>> on.
> Just to be clear in my mind and to make sure that I'm using the right stuff
> in my QMF mapping by the new internal model are you talking about the stuff
> that lives in:
> <qpid-root>/java/broker/src/main/java/org/apache/qpid/server/model
> Please say yes :-)

Yes :-)

> My QmfManagementAgent is built following the same general patterns as the
> management-http and management-jmx stuff in broker-plugins using the
> ManagementFactory and ManagementPlugin stuff. I've basically used the
> ConfigurationChangeListener interface methods to track changes to the
> server/model ConfiguredObjects and when childAdded is called I use that to
> create concrete QmfAgentData objects that wrap the equivalent
> ConfiguredObject instance. There's *largely* a one-to-one between those
> objects and their QMF equivalents though some of the properties/statistics
> need to be mapped.
> To be honest the hardest stuff I've had to cope with to date was due to the
> fact that qpid-config is a bit weird - it doesn't use object references
> correctly and has a hard coded ObjectId for the broker agent for method
> invocation (rather than discovering the broker object's ObjectId) so I had
> to make sure that my Agent copes with that.
>> Other than that I don't think there are *huge* gaps, just a number of
>> things done annoyingly differently (Gordon and I have tried to rectify
>> this in fits and starts over the years).  The QMF schema also tended
>> to make some assumptions based on how the C++ broker was modelled that
>> just didn't hold true for the Java Broker.
> If I'm doing it right from what I've seen the differences aren't so very bad
> with the new model. It's only taken me a couple of weekends to get to the
> state where both my QMF GUI and qpid-config can do useful things with the
> Java Broker.
> One of the biggest quirks is actually the Virtual Host stuff in the Java
> Broker. Although there are Virtual Host QMF Objects I'm not aware that the
> C++ Broker is really Virtual Host aware? I don't believe that things like
> qpid-config are either - are they?

Yeah - this is another biggie - the C++ broker doesn't really do
vhosts.  There are operatins on the QMF "broker" that actually relate
to the vhost on the Java, but there are some things that truely are
"broker" properties.

> What I've done in order to allow things like qpid-config to work is to
> prefix queues/exchanges relating to the non-default vhost in other words:
> [vhost:<vhost-name>/]<exchange-name> and [vhost:<vhost-name>/]<queue-name>
> e.g.
> qpid-config exchanges
> Type      Exchange Name                 Attributes
> ============================================================
> fanout    vhost:development/amq.fanout  --durable
> topic     vhost:localhost/amq.topic     --durable
> direct    vhost:localhost/amq.direct    --durable
> fanout    vhost:localhost/amq.fanout    --durable
> direct    vhost:localhost/test.direct   --durable
> headers   vhost:localhost/amq.match     --durable
> headers   vhost:development/amq.match   --durable
> topic     monkey1
> fanout    monkey3
> direct    vhost:development/            --durable
> direct    qmf.default.direct            --durable
> I need to look to ordering the getObjects() query response base on name to
> make this look neater, but it seems to be the most interoperable approach at
> the moment.
>> So, personally I'm mostly looking at AMQP 1.0 these days where there
>> isn't the Exchange/Queue division and selection/distribution is a
>> property of the links.
> Is there a good simple reference explaining this stuff (aside from poring
> over the AMQP 1.0 specs, which I've not done yet - or could you point me at
> the relevant subsection?). I'm a little nervous 'cause to date I've mostly
> been thinking around exchanges, queues and bindings between them. From what
> I'd understood qpid::messaging was intended to provide an abstraction that
> was agnostic to the AMQP version so I'm assuming that existing
> AddressStrings still ought to work with AMQP 1.0??

I'm not sure whether there's any really good summary documentation out
there.  In essence AMQP 1.0 only defines the semantics of reliable
message transfer between peers - it does very little in terms of
definition of the internal broker model.  It talks of sending messages
between sources and targets and defines some associated semantics
(like the existence of fiters/selectors)... but nothing about what a
broker might do in terms of internal routing.

AddressStrings (in the qpid::messaging sense) should be a purely
client side implementation detail, so if correctly implemented they
should still "work" with AMQP 1.0... but they aren't really something
an AMQP 1.0 broker would be aware of.

>>   As such I'm not sure I'd personally spend a
>> lot of time writing an XML exchange... but given the XML tooling
>> available in Java then I wouldn't imagine it would be a huge task.
> That's why I was curious about the XML exchange. I personally prefer the
> idea of relatively simple subscriptions with finer grained filtering using
> Message Selectors but that was never really an option with C++ broker and
> clients. You could do it with C++ broker and Java clients but the Message
> Selectors were implemented in the client runtime so it wasn't a very
> efficient approach. Andrew Stitcher has just announced initial Message
> Selector support on the C++ broker, so I'm very excited about that.

Yeah - I've not looked at Andrew's work in detail... other than to
note that as ever it is close but not identical to the Java
implementation ... I believe that the implementation is only as a
filter on the consumer from a queue... In the Java Broker we also
added the ability to add selectors to a binding between a queue and an
exchange (in fact only on the topic exchange, but this is something we
should extend to all exchanges).  Adding selectors to bindings is
necessary if you want to do pub/sub with selectors, else your
temporary subscription queues fill up with stuff that you are never
going to consume.

>> The larger task I would like to undertake is to make it possible to
>> just drop in jars containing new Exchange / Queue types in a pluggable
>> manner.
> That sounds really cool.
>> Speaking of AMQP 1.0, we're currently working on standardising
>> Management over AMQP 1.0.  I'd expect the first draft of this to come
>> out in the next month or so... covering the mechanisms but not
>> attempting to define any "schema" for manageable objects.  This is
>> probably where I'll be looking to expend any time I have relative to
>> management work (especially when coupled with AMQP 1.0 over WebSockets
>> and a pure JavaScript AMQP 1.0 implementation).
> Rob as you might have gathered given all the work I've done on QMF and the
> QMF GUI I'm *extremely* interested in anything that is afoot with respect to
> Management. I've put a huge amount of effort into that stuff so I'm clearly
> keen to make sure that changes don't break things (well too badly :-)) and
> to evolve things where possible. I guess that the same is true of the broker
> internals such as the
> <qpid-root>/java/broker/src/main/java/org/apache/qpid/server/model which
> seems to be fairly recent.
> I'd be really grateful if you could keep me in the loop on the current
> thinking on this.

Sure - we (the OASIS AMQP Technical Committee) have not published
anything yet, but should do in a couple of weeks.  I don't suppose you
work for a firm that is already an OASIS member, do you?

> Re "AMQP 1.0 over WebSockets and a pure JavaScript AMQP 1.0 implementation"
> that sounds pretty cool. How far down the road has this travelled? Would the
> approach be to go from AMQP 1.0 in a browser say via a WebSocket Server such
> as Node.js then via TCP sockets to a Broker.

Adding websockets as a transport to an existing broker is no big deal
- I knocked up a prototype for the Java Broker in about an hour using
Jetty.  I'm not overly familiar with library availability in the C/C++
world but I'd expect it should be similarly easy to integrate
websockets as a transport alternative to TCP for other languages too.

> I guess a pure JS Broker
> deployed into Node would be cool too. Is this likely to be limited to
> Node.js or would you expect the WebSocket bridge to be deployable on other
> Servers?
> Cheers,
> Frase

-- Rob

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org

To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

View raw message