qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > QMF Map Message Protocol
Date Fri, 05 Feb 2010 16:08:00 GMT
<html>
<head>
    <base href="http://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/1519/1/1/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background-color: white" bgcolor="white">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
     <h2><a href="http://cwiki.apache.org/confluence/display/qpid/QMF+Map+Message+Protocol">QMF
Map Message Protocol</a></h2>
     <h4>Page <b>edited</b> by             <a href="http://cwiki.apache.org/confluence/display/~tross">Ted
Ross</a>
    </h4>
     
          <br/>
     <div class="notificationGreySide">
         <h1><a name="QMFMapMessageProtocol-QMFv2MapMessageProtocol"></a>QMFv2
Map Message Protocol</h1>
<h2><a name="QMFMapMessageProtocol-Introduction"></a>Introduction</h2>

<p>This document describes the design of a proposed protocol for QMF based on map-messages
(offered by the new C++ and Python APIs as well as the existing JMS API).</p>

<p>If adopted, this new protocol will change the formats of the messages used by QMF
components to communicate.  It will also change some of the message exchange patterns.  It
will <b>not</b> significantly impact the console and agent APIs and is intended
to operate with applications that use the current QMF APIs.</p>

<p>Some highlights of the new design:</p>

<ul>
	<li>Current QMF message bodies are in packed binary formats.  While quite efficient,
this style of formatting makes it difficult to make changes to the format and content for
new features.  The proposed format is based on encoded maps (a.k.a. dictionaries, field-tables)
which are very easily extended and require less context to be useful.</li>
	<li>QMF currently requires the message broker to participate in the QMF protocol. 
The proposed protocol removes this requirement and will run properly on any AMQP message broker.</li>
	<li>QMF Agents currently publish periodic updates of their managed content to a globally
accessible topic.  This has security implications with regard to access to data.  This is
also inflexible in that updates to all data are sent at the same intervals.  The proposed
protocol removes the global publishing of data and introduces a subscription-query whereby
a console may request that an agent publish certain data at a certain interval to an indicated
target.  Such requests can be subject to access control and may be focused on only the data
that is needed for a particular application.</li>
	<li>The proposed protocol allows for more general use of data.  For example:
	<ul>
		<li>Free-form data, that has no object-identifier nor schema, can be transferred.
 This is useful for complex queries (joins, reports, etc.).</li>
		<li>Methods can be invoked against an agent in the absence of a managed object.</li>
	</ul>
	</li>
</ul>


<h2><a name="QMFMapMessageProtocol-QMFProtocol"></a>QMF Protocol</h2>

<h3><a name="QMFMapMessageProtocol-UseofMessageHeaders"></a>Use of Message
Headers</h3>

<h4><a name="QMFMapMessageProtocol-StandardMessageProperties"></a>Standard
Message Properties</h4>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Message Property</th>
<th class='confluenceTh'>Use</th>
</tr>
<tr>
<td class='confluenceTd'>correlation-id</td>
<td class='confluenceTd'>Used in request/response/indication sets to correlate responses
and indications to their request.</td>
</tr>
<tr>
<td class='confluenceTd'>reply-to</td>
<td class='confluenceTd'>Used in requests to indicate the address for the response.</td>
</tr>
<tr>
<td class='confluenceTd'>content-type</td>
<td class='confluenceTd'>'amqp/map' or 'amqp/list'</td>
</tr>
<tr>
<td class='confluenceTd'>user-id</td>
<td class='confluenceTd'>Supplied in a request if authentication/authorization at the
agent is appropriate.</td>
</tr>
<tr>
<td class='confluenceTd'>app-id</td>
<td class='confluenceTd'>'qmf2'</td>
</tr>
</tbody></table>

<h4><a name="QMFMapMessageProtocol-CustomApplicationHeaders"></a>Custom
Application Headers</h4>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Application Header Key</th>
<th class='confluenceTh'>Use</th>
</tr>
<tr>
<td class='confluenceTd'>method</td>
<td class='confluenceTd'>'request', 'response', or 'indication'.  This field describes
the message's role in a particular message-exchange pattern.</td>
</tr>
<tr>
<td class='confluenceTd'>qmf.opcode</td>
<td class='confluenceTd'>QMF-specific operation code (see list below).  The opcode defines
what content, if any, is to be found in the message body.</td>
</tr>
<tr>
<td class='confluenceTd'>qmf.content</td>
<td class='confluenceTd'>If the opcode is a data indication, this field indicates what
kind of data will be found in the message body.</td>
</tr>
<tr>
<td class='confluenceTd'>qmf.agent</td>
<td class='confluenceTd'>If this message is a data indication sent by an agent, this
field contains the agent's name.</td>
</tr>
</tbody></table>

<h4><a name="QMFMapMessageProtocol-QMFOpCodes"></a>QMF OpCodes</h4>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>qmf.opcode field</th>
<th class='confluenceTh'>Message Body Data Type</th>
<th class='confluenceTh'>Description</th>
</tr>
<tr>
<td class='confluenceTd'>_agent_locate_request</td>
<td class='confluenceTd'>QMF_QUERY_PREDICATE</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_agent_locate_response</td>
<td class='confluenceTd'>QMF_DATA</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_agent_heartbeat_indication</td>
<td class='confluenceTd'>QMF_DATA</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_query_request</td>
<td class='confluenceTd'>QMF_QUERY</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_query_response</td>
<td class='confluenceTd'>???</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_subscribe_request</td>
<td class='confluenceTd'>QMF_SUBSCRIBE</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_subscribe_cancel_request</td>
<td class='confluenceTd'>VOID</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_subscribe_refresh_indication</td>
<td class='confluenceTd'>VOID</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
<tr>
<td class='confluenceTd'>_data_indication</td>
<td class='confluenceTd'>List of &lt;qmf.content&gt;</td>
<td class='confluenceTd'>&nbsp;</td>
</tr>
</tbody></table>

<p>h4 QMF Content Types</p>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>qmf.content field</th>
<th class='confluenceTh'>Data Type</th>
<th class='confluenceTh'>Description</th>
</tr>
<tr>
<td class='confluenceTd'>_schema_package</td>
<td class='confluenceTd'>STRING</td>
<td class='confluenceTd'>Schema package name</td>
</tr>
<tr>
<td class='confluenceTd'>_schema_id</td>
<td class='confluenceTd'>SCHEMA_ID</td>
<td class='confluenceTd'>Schema class identifier</td>
</tr>
<tr>
<td class='confluenceTd'>_schema_class</td>
<td class='confluenceTd'>SCHEMA_CLASS</td>
<td class='confluenceTd'>Schema class definition</td>
</tr>
<tr>
<td class='confluenceTd'>_object_id</td>
<td class='confluenceTd'>OBJECT_ID</td>
<td class='confluenceTd'>Managed object identifier</td>
</tr>
<tr>
<td class='confluenceTd'>_data</td>
<td class='confluenceTd'>QMF_DATA</td>
<td class='confluenceTd'>Data, managed and/or described or free-form</td>
</tr>
<tr>
<td class='confluenceTd'>_event</td>
<td class='confluenceTd'>QMF_EVENT</td>
<td class='confluenceTd'>Event</td>
</tr>
</tbody></table>


<h3><a name="QMFMapMessageProtocol-MessageBodyMapFormats"></a>Message Body
Map Formats</h3>

<h4><a name="QMFMapMessageProtocol-SCHEMAID"></a>SCHEMA_ID</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>  SCHEMA_ID := { _package_name: STRING,
                 _class_name:   STRING,
                 _type:         _data | _event,
                 _hash_str:     STRING
               }
</pre>
</div></div>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Field</th>
<th class='confluenceTh'>Optional</th>
<th class='confluenceTh'>Description</th>
</tr>
<tr>
<td class='confluenceTd'>_package_name</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Package name (namespace) for the described class</td>
</tr>
<tr>
<td class='confluenceTd'>_class_name</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Name of the described class</td>
</tr>
<tr>
<td class='confluenceTd'>_type</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Class type: data or event</td>
</tr>
<tr>
<td class='confluenceTd'>_hash_str</td>
<td class='confluenceTd'>yes</td>
<td class='confluenceTd'>Hash string to distinguish different versions of a class</td>
</tr>
</tbody></table>

<h4><a name="QMFMapMessageProtocol-SCHEMACLASS"></a>SCHEMA_CLASS</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>  SCHEMA_CLASS := { _schema_id: SCHEMA_ID,
                    _values:    { EACH_ATTR_NAME: SCHEMA_PROPERTY | SCHEMA_METHOD },
                    _subtypes:  { EACH_ATTR_NAME: qmfProperty | qmfMethod }
                  }
</pre>
</div></div>

<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Field</th>
<th class='confluenceTh'>Optional</th>
<th class='confluenceTh'>Description</th>
</tr>
<tr>
<td class='confluenceTd'>_schema_id</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Identifier for this schema class</td>
</tr>
<tr>
<td class='confluenceTd'>_values</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Map of schema attribute names and either their property or
method descriptions.  The subtype defines whether an attribute is a property or a method.</td>
</tr>
<tr>
<td class='confluenceTd'>_subtypes</td>
<td class='confluenceTd'>no</td>
<td class='confluenceTd'>Map of subtype names ('qmfProperty' or 'qmfMethod') for each
attribute</td>
</tr>
</tbody></table>

<h3><a name="QMFMapMessageProtocol-Operations"></a>Operations</h3>

<h4><a name="QMFMapMessageProtocol-AgentAnnounce"></a>Agent Announce</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                      Topic                         Agent
             |                           |                             |
             |                           | &lt;---- Agent Indication ---- |
             | &lt;------------------------ |                             |
             |                           |                             |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-AgentLocate"></a>Agent Locate</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                            Topic                         Agent
             |                                 |                             |
             | ---- Agent Locate Indication -&gt; |                             |
             |                                 | --------------------------&gt; |
             |                                 |                             |
             | &lt;-------------------------------------- Agent Indication ---- |
             |                                 |                             |
             |                                 |                             |
             | ---- Agent Locate Indication -------------------------------&gt; |
             |                                 |                             |
             | &lt;-------------------------------------- Agent Indication ---- |
             |                                 |                             |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-SchemaQuery"></a>Schema Query</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                                  Agent
             |                                       |
             | ---- Schema Request ----------------&gt; |
             |                                       |
             | &lt;------------- Schema Indication ---- |
             | &lt;------------- Schema Indication ---- |             
             |                                       |
             | &lt;---------------------- Response ---- |
             |                                       |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-SchemaAnnounce"></a>Schema Announce</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                      Topic                         Agent
             |                           |                             |
             |                           | &lt;--- Schema Indication ---- |
             | &lt;------------------------ |                             |
             |                           |                             |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-GetQuery"></a>Get Query</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                                  Agent
             |                                       |
             | ---- Get Query Request -------------&gt; |
             |                                       |
             | &lt;--------------- Data Indication ---- |
             | &lt;--------------- Data Indication ---- |
             | &lt;--------------- Data Indication ---- |
             | &lt;--------------- Data Indication ---- |
             | &lt;--------------- Data Indication ---- |
             | &lt;---------- Last Data Indication ---- |
             |                                       |
             | &lt;------------------------ Result ---- |
             |                                       |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-CreateSubscriptionQuery"></a>Create Subscription
Query</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                Target                              Agent
             |                     |                                   |
             | ---- Create Subscription Request ---------------------&gt; |
             |                     |                                   |
             |                     | &lt;----------- Data Indication ---- |
             |                     | &lt;----------- Data Indication ---- |
             |                     | &lt;----------- Data Indication ---- |
             |                     | &lt;----------- Data Indication ---- |
             |                     | &lt;----------- Data Indication ---- |
             |                     |                                   |
             | &lt;------------------------------------------ Result ---- |
             |                     |                                   |
             |                     | &lt;----------- Data Indication ---- |
             |                     |                                   |
             |                     | &lt;----------- Data Indication ---- |
             |                     |                                   |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-CancelSubscriptionQuery"></a>Cancel Subscription
Query</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console               Target                              Agent
             |                     |                                   |
             | ---- Cancel Subscription Request ---------------------&gt; |
             |                     |                                   |
             | &lt;------------------------------------------ Result ---- |
             |                     |                                   |
             |                     | &lt;----------- Data Indication ---- |
             |                     | &lt;------ Last Data Indication ---- |
             |                     |                                   |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-RenewSubscriptionQuery"></a>Renew Subscription
Query</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                Target                              Agent
             |                     |                                   |
             | ---- Renew Subscription Request ----------------------&gt; |
             |                     |                                   |
             | &lt;------------------------------------------ Result ---- |
             |                     |                                   |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-InvokeMethod"></a>Invoke Method</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                                  Agent
             |                                       |
             | ---- Method Request ----------------&gt; |
             |                                       |
             | &lt;------------------------ Result ---- |
             |                                       |
</pre>
</div></div>

<h4><a name="QMFMapMessageProtocol-RaiseEvent"></a>Raise Event</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>          Console                Topic                               Agent
             |                     |                                   |
             |                     | &lt;---------- Event Indication ---- |
             | &lt;------------------ |                                   |
             |                     |                                   |
</pre>
</div></div>

<h3><a name="QMFMapMessageProtocol-MapBodyFormats"></a>Map Body Formats</h3>

<h4><a name="QMFMapMessageProtocol-QMFDataFormats"></a>QMF Data Formats</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>    qmfAgentLocate := {qmfQuery}

    qmfAgentIndicate := {qmfAgentInfo}

    qmfAgentInfo := agent_info: {name: (sstr),
                                 vendor: (sstr),
                                 product: (sstr),
                                 version: (sstr),   -- optional
                                 platform: (sstr),  -- optional
                                 arch: (sstr)}      -- optional

    qmfQuery := query: {what: qmfQueryTarget,
                        where: qmfQueryPredicate}  -- optional

    qmfQueryTarget := {schema_package: (void)} ||
                      {schema_id: (void)} ||
                      {schema: (void)} ||
                      {agent: (void)} ||
                      {object_id: qmfClassSpecifier} ||
                      {object: qmfClassSpecifier}

    qmfClassSpecifier := {class_name: (sstr),
                          package_name: (sstr),    -- optional
                          hash: (bin128)}          -- optional

    qmfQueryPredicate := {qmfCompare} ||
                         {and: listOf(qmfQueryPredicate)} ||
                         {or: listOf(qmfQueryPredicate)} ||
                         {not: qmfQueryPredicate}

    qmfCompare := list(eq, (sstr), amqpValue) ||
                  list(ne, (sstr), amqpValue) ||
                  list(lt, (sstr), amqpValue) ||
                  list(gt, (sstr), amqpValue) ||
                  list(le, (sstr), amqpValue) ||
                  list(ge, (sstr), amqpValue) ||
                  list(exists, (sstr)) ||
                  list(is-type, (sstr), amqpTypeCode) ||
                  list(is-subtype, (sstr), (sstr)) ||
                  list(re-match, (sstr), (lstr))

    qmfData := {values: mapOf(amqpValue),
                subtypes: mapOf(sstr) }    -- optional, keys must be in values

    qmfDescribedData := {(qmfData),
                         class_id: qmfClassId }

    qmfManagedData := {(qmfDescribedData),
                       object_id: (lstr),
                       create_time: (datetime),
                       delete_time: (datetime),  -- optional
                       update_time: (datetime) }

    qmfClassId := {package: (sstr),
                   name: (sstr),
                   hash: (bin128) }

    qmfObjectClass := {(qmfData[qmfProperty|qmfMethod]),
                       class_id: qmfClassId,
                       desc: (lstr),
                       superclass: qmfClassId }

    qmfProperty := {amqp_type: (amqpTypeCode),
                    access_rule: (sstr),
                    optional: (void),
                    unit: (sstr),
                    desc: (lstr),
                    subtype: (sstr),
                    discrete: (void),
                    min: (amqp_type),
                    max: (amqp_type),
                    maxlen: (uint32) }

    qmfMethod := {desc: (lstr),
                  arguments: mapOf(qmfArgument) }

    qmdArgument := {amqp_type: (uint8),
                    unit: (sstr),
                    desc: (lstr),
                    subtype: (sstr),
                    input: (void)
                    output: (void)
</pre>
</div></div>
     </div>
     <div id="commentsSection" class="wiki-content pageSection">
       <div style="float: right;">
            <a href="http://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
       </div>

       <a href="http://cwiki.apache.org/confluence/display/qpid/QMF+Map+Message+Protocol">View
Online</a>
       |
       <a href="http://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=5603679&revisedVersion=18&originalVersion=17">View
Change</a>
              |
       <a href="http://cwiki.apache.org/confluence/display/qpid/QMF+Map+Message+Protocol?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org


Mime
View raw message