Hi folks,
I'm revisiting this having got my IT back, comments inline...
Frase
On 29/03/13 13:29, Gordon Sim wrote:
> On 03/28/2013 08:52 PM, Bill Freeman wrote:
>> So it occurs to me to ask whether the correspondence between named queue
>> and object id survives broker restarts?
>
> I believe with QMFv2 and qpidd, the object identifiers for queues are
> formed from the package, the class and the queue's name. So although
> the management id is not persisted, it would be the same on recovery
> (as it is essentially the queue name).
According to the protocol spec
https://cwiki.apache.org/qpid/qmf-map-message-protocol.html ObjectId is
formed thus:
OBJECT_ID := { _agent_name: STRING,
_agent_epoch: NUMBER,
_object_name: STRING
}
_agent_name yes Name of the agent that is managing the referenced data
_agent_epoch yes Numeric epoch of the agent process. This number is
managed by the agent and is incremented each time the agent process
starts. This field is only present for /transient/ object IDs that must
not be the same for a given object across an agent restart. /Persistent/
object IDs must not include this field.
_object_name no Name of the data that uniquely identifies the data
within the context of the agent.
According to the QMF2 API spec
https://cwiki.apache.org/qpid/qmfv2-api-proposal.html "An object
identifier uniquely addresses a data object within the domain of its
managing agent. QMF represents the object identifier as an opaque string"
Clearly there's a bit of ambiguity between "opaque string" and the
fields specified in the protocol :-)
It's even weirder because despite the alleged opacity there's ambiguity
because the API has in SchemaObjectClass :
.set_id_names([name-list]): set the value of the order list of names to use when constructing
the object identifier.
In other words that's a mechanism used to set the _object_name. In my
Java Implementation for Agent I've got code that does:
ObjectId addr = object.getObjectId();
if (addr == null)
{
SchemaClassId classId = object.getSchemaClassId();
SchemaClass schema = _schemaCache.get(classId);
// Try to create an objectName using the property names
that have been specified as idNames in the schema
StringBuilder buf = new StringBuilder();
// Initialise idNames as an empty array as we want to check
if a key has been used to construct the name.
String[] idNames = {};
if (schema != null && schema instanceof SchemaObjectClass)
{
idNames = ((SchemaObjectClass)schema).getIdNames();
for (String property : idNames)
{
buf.append(object.getStringValue(property));
}
}
String objectName = buf.toString();
// If the schema hasn't given any help we use a UUID. Note
that we check the length of idNames too
// as a given named key property might legitimately be an
empty string (e.g. the default direct
// exchange has name == "")
if (objectName.length() == 0 && idNames.length == 0)
objectName = UUID.randomUUID().toString();
// Finish up the name by incorporating package and class names
objectName = classId.getPackageName() + ":" +
classId.getClassName() + ":" + objectName;
// Now we've got a good name for the object we create its
ObjectId and add that to the object
addr = new ObjectId(_name, objectName, _epoch);
object.setObjectId(addr);
}
So if I do setObjectId() on a QmfConsoleData instance before I call
addObject() it will explicitly use the ID I've set (I needed to do this
for the Java Broker Qmf plugin so the ObjectIds matched the assumptions
made by qpidlibtools), if I don't set the ObjectId but do set idNames in
the SchemaObjectClass then the objectName is formed from the
concatenation of the values of the specified properties and the package
and class name.
None of that is specified but seems to be consistent with what
qpidlibtools has done.
The bottom line is that in many ways I think it should be opaque, but
the ObjectId definitely has properties in the protocol and even the
_object_name isn't opaque, though I definitely think that should be but
I've implemented code that allows for the ambiguity of the specification
- if nothing else is specified you'll see I default to a UUID.
>
>> And I presume that deleting and recreating queues can change their
>> object
>> IDs?
>
> No, I believe deleting and recreating a queue will result in the same
> identifier being used since the name is the same.
I'm not convinced that that statement is correct. I think that it is
correct for the _object_name part of the ObjectId however as you'll note
the _agent_epoch forms part of the ObjectId. Although it's optional I'm
"pretty sure" that the broker passes the _agent_epoch value in QMF2
ObjectIds and I'm also "pretty sure" that the C++ broker increments the
epoch value each time it restarts.
I've not had time to check for certain, but I've got code in my Console
class that says:
// If there are any results
available after evaluating the query we deliver them
// via a SubscribeIndicationWorkItem.
// Before we send the WorkItem we
take a peek at the Agent Epoch value that forms
// part of the ObjectID and compare
it against the current Epoch value. If they
// are different we send an
AgentRestartedWorkItem. We *normally* check for Epoch
// changes when we receive
heartbeat indications, but unfortunately the broker
// ManagementAgent pushes data
*before* it pushes heartbeats. Its more useful
// however for clients to know that
an Agent has been restarted *before* they get
// data from the restarted Agent
(in case they need to reset any state).
if (objectEpoch > agent.getEpoch())
{
agent.setEpoch(objectEpoch);
agent.clearSchemaCache(); //
Clear cache to force a lookup
List<SchemaClassId> classes =
getClasses(agent);
getSchema(classes, agent); //
Discover the schema for this Agent and cache it
_log.info("Agent {} has been
restarted", agentName);
if (_discoverAgents &&
(_agentQuery == null || _agentQuery.evaluate(agent)))
{
_eventListener.onEvent(new AgentRestartedWorkItem(agent));
}
}
_eventListener.onEvent(
new SubscriptionIndicationWorkItem(
new
SubscribeIndication(consoleHandle, resultList))
);
It's been a while since I looked at that code, but I put it in place
when I wrote the Java port of qpid-queue-stat which uses the QMF2 query
subscription (I needed to emulate that on the client side by subscribing
to the broker data push). As the comment says it's useful to be able to
detect Agent restarts before the data from the restarted Agent gets sent
(qpid-queue-stat resets its Maps when it gets an Agent restart event).
So I'm really pretty sure that ObjectIds will be different when a broker
is restarted though I think that the _object_name part will be the same
as that is formed from the package, class and name property (well for
queues and exchanges it is for sure).
>
>> So, then, are Queue and Exchange the class (I'm guessing from the
>> code I've
>> read). Could you please explain packages?
>
> The package is really just a namespace. The goal of QMF was to be able
> to define all different sorts of schemas and a namespace qualification
> made this possible.
>
What Gordon says is pretty much it package is little more than a
namespace used to help disambiguate in case the same class is seen in
other Agents. It's worth looking through <qpid>/specs/management-schema.xml
Actually in the schema if you look at say Queue and Exchange you'll see:
<property name="name" type="sstr" access="RC" index="y"/>
I suspect that the "index=" bit is suggesting that that particular
property is being used to help form the _object_name, which ties in
rather with the _id_names stuff I mentioned earlier.
I'm still not convinced it's a good idea for ObjectIds to be
"interpretable/parsable" rather than opaque and as I say there's
definitely something of an ambiguity between what is stated in the API
document and what has been implemented in the qpidlibtools.
Not sure if any of the above actually helps :-) but hopefully it's
useful. When I get a moment I'll add some debug to some test code to
check on this, but I'm relatively sure what I've said is accurate from
memory and given the comments and logic in my own code.
HTH
Frase
|