qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: QMF and Broker Management
Date Thu, 05 Jan 2012 18:05:33 GMT
On 05/01/12 16:28, Robbie Gemmell wrote:
> I also think that if any moves are being done, I would go further than
> just bumping them into the extras directory. That still isnt all that
> visible or seperate from Qpid in itself, I think the core QMF
> components would collectively warrant at least sub project status with
> their own release timelines and website etc.
> Robbie
I sent much of this reply on the Java QMF2 API thread but I thought I'd 
add it here too.

I think I'm pretty much a "+1" too. I'd tend to agree with Ted, yourself 
and the others that it's a good idea to separate QMF from the Qpid 
namespace and consider QMF to be a separate and layered capability that 
adds value to Qpid. That's exactly the case with the Java API 
implementation, which is really to all intents and purposes just a JMS 

My one reservation is that if QMF got separated there may be a danger 
that it could become a bit of a second class citizen, I'd be very keen 
to avoid that happening.

As it happens one of the reasons I started down the path of the Java 
QMF2 API implementation was because the original Java QMF stuff seemed 
very broken (and it was mentioned that it was not supported/maintained 
and up for removal from the code base). Given my experience with the 
QMF2 stuff I suspect where the original Java QMF stuff is most likely to 
be broken is in the String handling. I've mentioned in several threads 
lots of nasty interoperability issues due to C++ binary strings (as 
opposed to utf8 encoded strings), which get mapped into byte[]. I've put 
in a lot of defensive coding to cater for this.

If I'm honest I think that my preference is to have QMF initially as a 
sub-project under the qpid umbrella so that there's some initial 
monitoring from the core qpid community to ensure that there's 
sufficient "traction". IMHO QMF is too important to qpid to be allowed 
to stagnate - indeed I'd like to see a whole lot of additional 
development done.

Part of my motivation for the Java QMF stuff was to try and kick start 
some of this. One of the things that you often see with other messaging 
providers are fancy consoles and the like, but with qpid (particularly 
the C++ broker) things are a bit more limited to command line utilities, 
which may put people off.

I'd very much like to see the QMF stuff tidied up in the C+ broker code 
base so I like Ted's suggestion "move the C++ implementation of qmf2 out 
of the qpid tree". When I was trying to figure out what was going on I 
ended up down a lot of blind alleys because of the somewhat confusing 
structure of the code base as-is.

I guess the elephant in the room is what to do about QMF1. Is the 
intention to deprecate this? Clearly the standard tools could be 
rewritten using QMF2API  (or indeed just use the protocol), but what 
about people who've written bespoke consoles/agents using the old 

IMHO it rather waters down Ted's proposal to move out QMF2 but keep the 
old QMF1 stuff in the core code base - indeed it could make things very 

One possibility that I've been musing over is that if we layer QMF we 
could provide a "wrapper" at the edges to "transcode" to the QMF1 protocol.

Another option is to provide QMF1 and QMF2 APIs but under the hood they 
both talk the QMF2 protocol. I guess that'll break down if anyone has 
written consoles/agents that speak the old QMF1 protocol directly rather 
than using the API, but I'd suspect that is fairly unlikely given how 
much of a faff that would be.


> On 4 January 2012 18:51, Rob Godfrey<rob.j.godfrey@gmail.com>  wrote:
>> On 4 January 2012 18:55, Ted Ross<tross@redhat.com>  wrote:
>>> Users and Devs,
>>> I'd like to make a proposal and start a discussion about the future of QMF
>>> and Qpid broker management.
>>> QMF (Qpid Management Framework) started out as a way to remotely manage
>>> the Qpid C++ broker using AMQP messaging.  There was an agent embedded in
>>> the broker and a console API written in Python.  It was then expanded for
>>> more general purpose use when an agent library and API were developed so
>>> developers could provide QMF manageability to their software components.
>>> There has been quite a bit of evolution including new APIs and even a new
>>> protocol based on map/list-encoded messages.  One of the important changes
>>> that occurred with the new protocol (called qmf2) was that QMF became
>>> purely layered over AMQP messaging.  The original protocol required the
>>> participation of the broker to assign addresses, to track agents, and to
>>> cache schema information (didn't scale well, didn't work in multi-broker
>>> environments, had multiple protocol issues, wreaked havoc with clustering).
>>> The QMF code is embedded in the "qpid" namespace because older versions
>>> were tightly coupled to the broker code.  Now that the coupling has been
>>> reduced (consisting of the public messaging API), it is possible to move
>>> QMF out of the "qpid" namespace and allow it to be a separate component,
>>> with its own build and release artifacts.
>>> I would like to propose that we:
>>> 1. move the C++ implementation of qmf2 out of the qpid tree and into
>>>    the "extras" subdirectory (where the Python implementation is),
>>> 2. move the swig bindings (Python, Ruby, etc.) into extras as well, and
>>> 3. deprecate the old qmf components.
>>> The old components are in:
>>>   * cpp/{include,src}/console
>>>   * cpp/{include,src}/agent
>>>   * cpp/{include,src}/qmf/engine
>>>   * cpp/bindings/qmf
>> +1 I think this will be beneficial both for QMF and Qpid Core development.
>> Should we be thinking of spinning QMF off as a separate project in its own
>> right?  I'm guessing that if the coupling is loose enough there is no need
>> to tie release schedules of QMF to the release cycles of the rest of the
>> project?
>> The last part of the proposal is to remove the dependency that the qpid
>>> tools (qpid-config, qpid-stat, qpid-route, etc.) have on the Python QMF
>>> library.  If you haven't noticed, these tools run fairly slowly, especially
>>> when grouped in large numbers in a script file.  This is because QMF
>>> (version 1) has a significant amount of handshake that occurs on connection
>>> setup.  Since the tools don't need agent-discovery or schema-introspection,
>>> they can operate much more simply by sending and receiving properly
>>> formatted messages to and from the broker agent.  I prototyped this with
>>> qpid-stat and found it to be visually instantaneous in its response time.
>>>   It also reduced the number of queues and bindings related to the
>>> management session to one.
>> Can you confirm that you are going to be testing this with the Java Broker
>> as well as the C++?
>> Happy to help if there are any issues that crop up on the Java side of this.
>>> Fraser Adams has contributed a Java implementation of the new QMF
>>> protocol.  It makes sense to me that this should be included with the C++
>>> and wrapped components that I propose moving into "extras".
>> Agreed.
>> Cheers,
>> Rob
>> Thoughts?
>>> Regards,
>>> -Ted
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

View raw message