qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Freeman <ke1g...@gmail.com>
Subject Re: Questions from a novice
Date Fri, 29 Mar 2013 16:35:14 GMT
On Fri, Mar 29, 2013 at 9:29 AM, Gordon Sim <gsim@redhat.com> wrote:

> On 03/28/2013 08:52 PM, Bill Freeman wrote:
>
>> So it occurs to me to ask whether the correspondence between named queue
>> and object id survives broker restarts?
>>
>
> I believe with QMFv2 and qpidd, the object identifiers for queues are
> formed from the package, the class and the queue's name. So although the
> management id is not persisted, it would be the same on recovery (as it is
> essentially the queue name).


I'm actually using RH MRG, but that seems to be little different from
upstream Qpid.  I'm certainly seeing v2 qmf queues mentioned in the broker
__dict__.  Please say if I'm making unwarranted assumptions here.  I wanted
my code to be as generic as possible, so I'm trying to avoid depending on
any RH specializations.

>
>
>  And I presume that deleting and recreating queues can change their object
>> IDs?
>>
>
> No, I believe deleting and recreating a queue will result in the same
> identifier being used since the name is the same.


That's good to know.  I have been using str(record.objectId()), where
record is the second argument (third if you count self) to an objectProps()
method on a sub-class of qmf.console.Console.  I'm guessing that the final
number in this hyphen separated string of digits, where the last number
tends to be the only multi-digit one, and comes from the objectName field
of qmf.console.ObjectId instances, is a hash of the information that you
mention?
Or am I confusing two separate meanings of "object ID"?
If so, how to I get the ID or which you speak.
If not, doesn't hashing to such a small number have collision potential?

>
>
>  So, then, are Queue and Exchange the class (I'm guessing from the code
>> I've
>> read).  Could you please explain packages?
>>
>
> The package is really just a namespace. The goal of QMF was to be able to
> define all different sorts of schemas and a namespace qualification made
> this possible.
>

I just didn't fine descriptions of what these were in the docs, though I'm
calling functions that require them.  If there's somewhere I should have
looked, I'd appreciate a pointer.

>
> I'm not 100% sure if I've understood your questions or answered them
> satisfactorily. Please feel free to follow up. There are also others on
> this list with more in-depth knowledge of QMF.


I think that you're right on track.  I should probably give a bit of
background on my project as a context for understanding my questions.  Or
ignore the rest of this if not interested:

I'm producing a monitoring tool to help spot issues in a federation having
many queues.  This may sound similar to Cumin, but we want displays in
different forms (bar graphs of queue message count compared to some defined
"full" constraint, color changes to attract the administrator's eye to
queues that are "in trouble", popups to allow browsing messages, looking at
bindings, move messages from queue to queue, group queues on the page in
configurable ways...  I looked at modifying cumin, but:
 1. It seems a bit complex - I think that I can write my own faster;
 2. we don't want to use  PostgreSQL (I like pg, but it's not on the
approved tools list);
 3. Cumin has a lot of features we don't need; and
 4. I've caught Cumin missing updates, i.e.;, I spout or drain, sometimes
the queue info shows up within 10 seconds (update/heartbeat rate), but
sometimes it takes another change or two before it shows up.

I have a strawman running just showing queue fullness with a few percent of
full color changes, and now I'm trying to move on to cover the federation.
I wound up with a Cumin like architecture:  A python based qmf console
process that publishes updates to a redis pubsub channel, subscribed to by
a tornado based web server, which pushes them on out to browser based
clients using WebSockets.  There's a custom jQueryUI widget per queue in
the browser that handles display and provides for a context menu.
WebSockets is inherently bi-directional, so I have another redis pubsub
channel for things like commands to move messages, requests to grab a
browse of a queue, get bindings, etc.  I'm using tornado because it
supports WebSockets out of the box, and there's a redis extension for it
using its event loop, so the web server manages to eschew threads.  The
architecture allows for load balancing by having multiple web servers
and/or multiple qmf console processes (though I intend that one qmf console
process be able to handle multiple browser connections, since the
underlying code seems to support that).

Bill

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message