qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: Questions from a novice
Date Mon, 01 Apr 2013 19:48:41 GMT

On 1 Apr 2013, at 19:56, Bill Freeman <ke1g.nh@gmail.com> wrote:

> I believe that I fall into the data push camp, even though I may not be
> using the modern means of achieving it.  I create my qmf.console.Session
> instance with rcvObjects = True.  Then, after an initial one copy of
> everything, I'm only sent updates if anything changes, and then at most
> once every 10 seconds.  I don't want to poll.  So maybe the flag works
> (though I have other concerns about a broker wide, rather than connection
> or even session specific setting).
That's definitely a QMF1 API that you are using then, there are actually two python QMF1 APIs
one uses getObjects() calls and the other has a notification type API. The 10 second interval
can be charged by a broker switch BTW I can't recall exactly what it is off the top of my
head, but if you do qpidd -h it should hopefully be obvious.

As an aside you mentioned previously about cumin "missing" updates, possibly that's true,
but it shouldn't really matter and your application should probably be designed to be tolerant
of this sort of thing - the updates of things like queueDepth, msgEnqueue, msgDequeue etc.
are all "incremental" so there's no concept of "resetting" values a client has to track an
initial value and deltas, it's worth pointing out though that timestamps are sent with each
update, so if an update has been delayed in transit you can use the update time stamp to know
the timestamp from the broker's perspective.

> Whatever its called, I need some kind of key to disambiguate objects.  I
> need them to be consistent across restarts of the broker, and maybe even
> across some minor reconfiguration of the broker that is outside of my
> control (such as the deletion and/or addition of queues, exchanges, and
> bindings).  Since I will be talking to multiple brokers, I need to guard
> against the possibility that a key could be repeated in a second broker
> (particularly when multiple brokers are federated to duplicate efforts as a
> fallback if one crashes).
It has been a while since I looked at the guts of QMF2 ObjectIds as I say I tend to treat
them as opaque. I think that Gordon's previous comments about V2 OIDs looked about right,
but I'd need to have a proper look. As he said V1 and V2 OIDs are rather different.

In V2 there's an epoch value that changes across broker restarts, I think that only applies
to the Agent name and not the OIDs, but it has been a while, I'll have a proper look when
I get IT back if you can bear with me.

You mention being consistent across broker restarts, I'm assuming you are talking about persistent
queues? Clearly non-persistent queues will be different objects. I'm not sure whether the
broker treats recreated persistent queues as the same objects each restart, possibly but I
can't be sure, it might not even be the right thing to do that, if you did the same thing
with say an in memory object reference they would definitely be at different addresses and
TBH that's usually how I view ObjectIds - as a way of describing an address at a point in
time. I'll need to take a prober look to ascertain the current truth though.

> I'm much better at Python than I am at Java, and my web server (which
> doesn't speak QMF anyway, being isolated by REDIS) is tornado, a python
> based tool.  
Fair enough if you are more familiar with python, but I really really would recommend moving
away from QMF1 and perhaps use the lib tool API Gordon mentioned earlier. As I mentioned before
management is likely to be in for something of a reshuffle, I think it's likely that there
will be attempts to make transitioning from QMF2 as easy as possible, but that probably won't
hold for QMF1.

> And Rest doesn't sound promising for getting updates pushed to
> me.
So you may be right, though QMF2 has a getWorkItem() call which can be blocking, that is used
in QMF2 in place of a pure asynchronous model as there was talk in the spec of avoiding threading
issues. I implemented this in REST via a long-polling call which works pretty well, my GUI
actually works by getting notified on heartbeats and other asynchronous events such as queueDeclare
etc. when triggered it then does a synchronous getObjects() call - via REST which ultimately
calls my Java getObjects(). I won't get into the relative pros and cons of long-polling versus
WebSockets, I generally agree that the latter is the way to go, but I needed wide browser
support and TBH I need to get a bit more up to speed on WebSockets, it's definitely on my
list of things to do, but the current approach works pretty well, and is arguably a pretty
good mapping to the underlying API call.

>> One last thing, there is some discussion beginning about the future of
>> management, the eventual intention is to move away from QMF towards a new
>> AMQP management API, however that is in the very early stages so at the
>> moment QMF is the only game in town for C++ broker management. There are a
>> few of us very keen to make sure that transition to the new API is as
>> painless as possible, but as I say it's very very early days and is mainly
>> said to try to ensure that you bear it in mind with your application. I'd
>> very definitely avoid QMF1 though QMF2 is a better bet.
> And my target brokers, which I don't get to specify, are C++ brokers.

These support QMF1 and QMF2, but from a client perspective I'd really recommend avoiding QMF1
if you possibly can.

The data push stuff is supported in both QMF1 and QMF2 formats. The QMF2 API allows a type
of query subscription, but the C++ broker doesn't actually support this :-( I actually managed
to fake this in my Java implementation by doing a data subscription and filtering on the client
side to implement the query API. That said it might have been overkill, I went to town implementing
the full QMF2 API, but it hasn't really taken off and the python lib tools stuff, although
it implements the QMF2 protocol doesn't really implement a documented API. 

The lack of standardisation of QMF2 and the hitherto lack of support on the Java broker is
one of the things driving the moves to try and create a properly standardised AMQP 1.0 management
protocol and API.

I've been working on QMF2 support for the Java broker and it's something of a mission of mine
to try and improve the overall cohesion of AMQP management, that was one of the key drivers
behind the GUI I put together and the work I'm starting on the Java broker.

I'll try to give a bit more detail later in the week on some of this, some of it will require
me to have a bit of an eyeball at some code.


To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

View raw message