qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Harvey (Confluence)" <conflue...@apache.org>
Subject [CONF] Apache Qpid > AMQP 1.0 JMS Client Requirements and Design
Date Thu, 22 Aug 2013 09:47:01 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/en/2176/1/21/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/qpid/AMQP+1.0+JMS+Client+Requirements+and+Design">AMQP
1.0 JMS Client Requirements and Design</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~phil@philharveyonline.com">Phil
Harvey</a>
    </h4>
        <div id="versionComment">
        <b>Comment:</b>
        removed HTML generation from phase 0 because we don't have any content yet<br />
    </div>
        <br/>
                         <h4>Changes (1)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >* Build tool configuration <br>*
CI server job <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">*
Automated HTML documentation generation. <br></td></tr>
            <tr><td class="diff-unchanged" > <br>h5. Phase 1 <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <p>This page describes the requirements and initial design of the new Qpid JMS
Client that supports AMQP 1.0 (simply referred to as the "Qpid JMS Client" below).</p>

<div>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Requirements'>Requirements</a></li>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Functionalrequirements'>Functional
requirements</a></li>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Phase0'>Phase 0</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Phase1'>Phase 1</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Phase2'>Phase 2</a></li>
</ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Nonfunctionalrequirements'>Non-functional
requirements</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Outofscope'>Out of scope</a></li>
</ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Design'>Design</a></li>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Layers'>Layers</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-PublicAPI'>Public API</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Locking'>Locking</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-I%2FOlayer'>I/O layer</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Logging'>Logging</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Testingstrategy'>Testing
strategy</a></li>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Testtypes'>Test types</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-Moduletests'>Module tests</a></li>
<ul>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-InprocessDriverimplementation'>In-process
Driver implementation</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-TestAmqpPeerimplementation'>TestAmqpPeer
implementation</a></li>
    <li><a href='#AMQP1.0JMSClientRequirementsandDesign-SASL'>SASL</a></li>
</ul>
</ul>
</ul>
</ul></div>

<h3><a name="AMQP1.0JMSClientRequirementsandDesign-Requirements"></a>Requirements</h3>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Functionalrequirements"></a>Functional
requirements</h4>

<ul>
	<li>AMQP 1.0</li>
	<li>JMS 2
	<ul>
		<li>JMS 1.1 support will be added in the future if demand requires.</li>
	</ul>
	</li>
	<li>ONGOING: Contribute to the JMS mapping work within the OASIS AMQP Binding &amp;
Mappings TC</li>
</ul>


<p>The following sub-sections allocate the functional requirements to several sequential
project phases. </p>

<h5><a name="AMQP1.0JMSClientRequirementsandDesign-Phase0"></a>Phase 0</h5>

<p>This is the initial set-up of the basics. </p>

<ul>
	<li>Open and close a connection. Includes writing automated tests.</li>
</ul>


<p>This will implicitly require work on:</p>

<ul>
	<li>Project folder structure</li>
	<li>Build tool configuration</li>
	<li>CI server job</li>
</ul>


<h5><a name="AMQP1.0JMSClientRequirementsandDesign-Phase1"></a>Phase 1</h5>

<ul>
	<li>Synchronous, Auto-Ack style 'simple client'</li>
	<li>Main operations on JMS Connections, Sessions, Consumers, Producers, Queues, Message
(Text?).
	<ul>
		<li>Include closing/stopping/deleting where applicable.</li>
	</ul>
	</li>
	<li>Configuration
	<ul>
		<li>Ability to configure settings such as pre-fetch etc
		<ul>
			<li>TODO per connection or per consumer or both? If the latter then not sure how
to pass this setting in (address options?), given that JMS API doesn't offer an obvious way
to do it.</li>
		</ul>
		</li>
		<li>TODO define how JNDI configuration will work</li>
	</ul>
	</li>
	<li>SASL</li>
	<li>Correct handling of non-happy path scenarios, namely:
	<ul>
		<li>Unexpected errors</li>
		<li>Failures that are nevertheless AMQP-compliant, e.g. the Broker spontaneously sends
a Close or End frame.</li>
		<li>Timeouts due to connectivity failure etc. Necessary because of the asynchronous
nature of AMQP.</li>
		<li>Interruption of application-owned threads.</li>
	</ul>
	</li>
</ul>


<h5><a name="AMQP1.0JMSClientRequirementsandDesign-Phase2"></a>Phase 2</h5>

<ul>
	<li>SSL</li>
	<li>Transactions</li>
	<li>Client Ack</li>
	<li>Selectors</li>
	<li>JNDI</li>
	<li>MessageListener.onMessage()</li>
	<li>Durable Subscriptions</li>
	<li>JMS 2.0 features:
	<ul>
		<li>Asynchronous send</li>
		<li>Shared Topic Subscriptions</li>
	</ul>
	</li>
</ul>


<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Nonfunctionalrequirements"></a>Non-functional
requirements</h4>

<ul>
	<li>Logging
	<ul>
		<li>Should play nicely with application logging, e.g. allow the application to plug
in its choice of third party logging framework, which would also be passed through to Proton
if possible. See  <a href="https://issues.apache.org/jira/browse/PROTON-343" class="external-link"
rel="nofollow">PROTON-343 - "Add a pluggable Proton logging layer"</a>.</li>
		<li>Should allow control of logging thresholds for categories including:
		<ul>
			<li>A specific object, e.g. a Connection</li>
			<li>A logical function, e.g. network I/O</li>
		</ul>
		</li>
	</ul>
	</li>
	<li>Initially will only run on Java 1.7 (because some JMS 2 features require Java 1.7).</li>
</ul>




<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Outofscope"></a>Out of
scope</h4>

<ul>
	<li>AMQP 0-x</li>
	<li>ADDR and BURL addressing formats</li>
	<li>Failover. Instead, this will be implemented in the future either entirely below
or entirely above the client code.</li>
	<li>Accepting legacy system properties (unelss they happen to match what we would choose
now).</li>
</ul>



<h3><a name="AMQP1.0JMSClientRequirementsandDesign-Design"></a>Design</h3>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Layers"></a>Layers</h4>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>+================================+
|
| JMS
|
| Implementations of javax.jms
|
+================================+
               |
               |
              \|/
+================================+
|
| AMQP
|
| Wrappers around Proton's engine
| classes, primarily to add locking
|
+================================+
            |                 |
            |                 |
           \|/                |
+======================+      |
|       Proton         |      |
|                      |      |
| Message |  Engine    |      |
|         |            |      |
+=========+============+      |
                    /|\       |
                     |        |
                     |       \|/
                   +====================
                   |
                   | JMS Driver interface
                   |
                   +=====================
                            /|\
                             |
                        +--------------+
                        |              |
                        |              |
                        |              |
                  +===========+        +==============
                  |                    |
                  |  Socket            |
                  |  driver            | Another driver,
                  |  (might use        | e.g. in-VM
                  |  Proton's Driver)  |
                  |                    |
                  +===========+        +==============
                    |
                    | TCP/IP
                    |
                   \|/
+================================+
|
| Broker
|
+================================+

</pre>
</div></div>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-PublicAPI"></a>Public
API</h4>

<ul>
	<li>Goals:
	<ul>
		<li>Minimal public API</li>
		<li>Possible to write JMS applications that have a compile-time dependency on the
javax.jms interfaces and not on Qpid-specific classes.</li>
		<li>Minor releases maintain backwards compatibility between minor releases.</li>
	</ul>
	</li>
</ul>


<p>The public API consists of:</p>
<ul>
	<li>The JMS interfaces (probably provided by a third party library such as <a href="http://mvnrepository.com/artifact/geronimo-spec/geronimo-spec-jms"
class="external-link" rel="nofollow">Geronomio Spec</a>).</li>
	<li>The constructor(s) for Qpid's implementation of javax.jms.ConnectionFactory and
its sub-interfaces.</li>
	<li>Qpid's javax.naming.spi.InitialContextFactory implementation.</li>
</ul>


<p><b>Note that the logging output and logging category names also constitute
a public interface</b>.</p>

<p>In the future, the public API may need to be extended or more precisely specified
(e.g. to include details of the mapping between JMS and AMQP when using MapMessage and ObjectMessage
etc).</p>


<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Locking"></a>Locking</h4>

<ul>
	<li>Goals:
	<ul>
		<li>Minimise the number of locks</li>
		<li>Define the correct locking order</li>
	</ul>
	</li>
</ul>


<p>Threads fall into two categories:</p>
<ol>
	<li>Application threads using the JMS API. Only use the top half of the Proton API.</li>
	<li>One or more driver threads for I/O. Managed by the JMS client. Only use the bottom
half of the Proton API.</li>
</ol>


<p>For a given connection:</p>

<ol>
	<li>State shared by multiple application threads (namely the state of the objects in
the JMS layer) is guarded by the JMS ConnectionLock.</li>
	<li>State shared by the application and driver threads is guarded by the AmqpConnection
lock. This shared state is:
	<ol>
		<li>A small number of flags the application and driver threads use to indicate to
each other that "something has changed".</li>
		<li>The Proton objects. This sharing occurs inside Proton but needs to be guarded
by the JMS client because Proton itself is not thread-safe.</li>
	</ol>
	</li>
</ol>


<p>Synchronous operations must follow the locking scheme indicated by the following
pseudo-code:</p>

<ul>
	<li>jmsObject.doStuff
	<ul>
		<li>obtain <b>ConnectionLock</b></li>
		<li>amqpObject.doAmqpStuff
		<ul>
			<li>obtain <em>amqpConnection lock</em></li>
			<li>protonObject.doProtonStuff</li>
			<li>release <em>amqpConnection lock</em></li>
		</ul>
		</li>
		<li>JmsConnection.stateChanged (event notification, e.g. to wake up the driver so
that I/O occurs)</li>
		<li>release <b>ConnectionLock</b></li>
	</ul>
	</li>
</ul>


<p>Where operations need to wait for a "remote" operation to complete, they must follow
the scheme indicated by the following pseudo-code:</p>

<ul>
	<li>jmsObject.doRemoteStuff
	<ul>
		<li>obtain <b>ConnectionLock</b></li>
		<li>...</li>
		<li>JmsConnection.waitUntil(predicateObject)
		<ul>
			<li>obtain <em>amqConnection lock</em></li>
			<li>amqConnection.wait() (the Driver thread calls amqConnection.notifyAll())</li>
			<li>...</li>
			<li>evaluate predicate</li>
			<li>...</li>
			<li>release <em>amqConnection lock</em></li>
		</ul>
		</li>
		<li>release <b>ConnectionLock</b></li>
	</ul>
	</li>
</ul>


<p>This ensures that when the predicate object is invoked the thread will already possess
both the JmsConnection lock and the AmqpConnection lock.</p>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-I%2FOlayer"></a>I/O layer</h4>

<ul>
	<li>Goals:
	<ul>
		<li>Encapsulate any 3rd party library as much as possible</li>
		<li>Allow transport to be easily swapped out to allow, for example:
		<ul>
			<li>in-VM operation</li>
			<li>WebSockets</li>
			<li>SCTP</li>
			<li>Accepting Connection. Useful if the client is behind a firewall that forbids
outbound connections.</li>
		</ul>
		</li>
	</ul>
	</li>
</ul>


<p>TODO - design I/O<br/>
Use Netty?</p>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Logging"></a>Logging</h4>

<ul>
	<li>Goals:
	<ul>
		<li>Consistent use of logging levels</li>
		<li>TODO more...</li>
	</ul>
	</li>
</ul>


<p>TODO</p>

<h4><a name="AMQP1.0JMSClientRequirementsandDesign-Testingstrategy"></a>Testing
strategy</h4>

<p>Goals:</p>
<ul>
	<li>Running all the tests should be fast (it should be exceptional for a test to take
more than five seconds).</li>
	<li>Tests operate at the correct level for their test type (see next section). Specifically:
	<ul>
		<li>Simple logic bugs cause unit tests to fail.</li>
		<li>Errors in collaboration logic cause module tests to fail.</li>
	</ul>
	</li>
	<li>The following third-part tests will also be run against the client:
	<ul>
		<li>Official JMS 2 TCK.</li>
		<li>Joram.</li>
	</ul>
	</li>
</ul>


<h5><a name="AMQP1.0JMSClientRequirementsandDesign-Testtypes"></a>Test types</h5>
<p>We will write a number of each of the following tests.</p>
<ul>
	<li>Unit test:  class that attempts to test an individual class. Some unit tests will
incidentally tests several other closely related classes too.</li>
	<li>Module test: a class that attempts to test the full JMS client, but using a stub/mock
etc instead of a broker. In most cases, we want to test that:
	<ul>
		<li>Calling specific JMS methods causes the correct AMQP to be sent, and</li>
		<li>The client correctly handles AMQP sent by its peer.</li>
	</ul>
	</li>
	<li>System test: a class that tests the behaviour of the JMS client when communicating
with a real peer such as the Qpid Broker. Assertions will be written mostly in terms of the
JMS API. We may re-use some of the existing Qpid system tests.</li>
</ul>


<h5><a name="AMQP1.0JMSClientRequirementsandDesign-Moduletests"></a>Module
tests</h5>

<p>The following diagram shows how module tests will work:</p>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>+================================+
|
| JUnit test
|
| 1. Create an in-process TestAmqpPeer
| 2. Set up TestAmqpPeer behaviour and expectations
| 3. Call some JMS methods
| 
+================================+
   |       |             |
   |       |             |
   |       |            \|/
   |       |    +================================+
   |       |    |
   |       |    | JMS client
   |       |    |
   |       |    +================================+
   |       |                |                 |
   |    latch/              |                 |
   |     unlatch etc        |                 |
   |       |               \|/                |
   |       |    +======================+      |
   |       |    |       Proton         |      |
expect/    |    |                      |      |
 assert    |    | Message |  Engine    |      |
   |       |    |         |            |      |
   |       |    +=========+============+      |
   |       |                /|\               |
   |       |                 |                |
   |      \|/               \|/              \|/
   |    +=======================================
   |    |
   |    | In-process Driver
   |    |
   |    +=======================================
   |                      /|\
   |                       |
   |                       | Bytes
   |                       |
  \|/                     \|/
+===================================+
|
| TestAmqpPeer
|
+===================================+
</pre>
</div></div>

<h6><a name="AMQP1.0JMSClientRequirementsandDesign-InprocessDriverimplementation"></a>In-process
Driver implementation</h6>
<ul>
	<li>Should use a threading model that is as similar as possible to the TCP/IP driver,
to make the test as realistic as possible.</li>
	<li>Should expose methods to give the test a way to control, for a given connection,
the relative order of (1) events in the "application" thread (typically the main test thread)
and (2) events in the driver thread. This should reduce the number of race condition bugs
in our code. We will probably implement this control using techniques such as:
	<ul>
		<li><a href="http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html"
class="external-link" rel="nofollow">Latches</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CyclicBarrier.html"
class="external-link" rel="nofollow">CyclicBarrier</a>s etc.</li>
		<li>Controlling the chunking of the byte production/consumption by both the In-process
Driver and the TestAmqpPeer.</li>
	</ul>
	</li>
</ul>


<h6><a name="AMQP1.0JMSClientRequirementsandDesign-TestAmqpPeerimplementation"></a>TestAmqpPeer
implementation</h6>
<p><b>Each test will give behaviour to a TestAmqpPeer</b>. This behaviour
will mostly be expressed in terms of AMQP frames. Here are prose examples (we don't yet know
how these will be expressed in code)</p>

<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Expected frame</th>
<th class='confluenceTh'>Frame to respond with</th>
</tr>
<tr>
<td class='confluenceTd'>An Open frame with container-id "xyz"</td>
<td class='confluenceTd'>The following canned Open frame: ...</td>
</tr>
</tbody></table>
</div>


<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Expected frame</th>
<th class='confluenceTh'>Frame to respond with</th>
</tr>
<tr>
<td class='confluenceTd'>An Open frame with container-id "xyz"</td>
<td class='confluenceTd'>The following sequence of frames representing a refused connection</td>
</tr>
</tbody></table>
</div>


<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'>Expected frame</th>
<th class='confluenceTh'>Frame to respond with</th>
</tr>
<tr>
<td class='confluenceTd'>Any Open</td>
<td class='confluenceTd'>Canned Open frame: ...</td>
</tr>
<tr>
<td class='confluenceTd'>Any Begin</td>
<td class='confluenceTd'>Canned Begin frame: ...</td>
</tr>
<tr>
<td class='confluenceTd'>An Attach matching the following criteria: ...</td>
<td class='confluenceTd'>The following Attach: ...</td>
</tr>
</tbody></table>
</div>


<p>In order to increase our confidence in the AMQP interoperabilty of the JMS client,
we want to avoid using the same Proton stack in the test peer as in the client. Therefore,
<b>Decoding and encoding will be done using proton-api's Data class.</b> The AmqpTestPeer
will minimise its use of other Proton classes.</p>

<h6><a name="AMQP1.0JMSClientRequirementsandDesign-SASL"></a>SASL</h6>
<p>Most tests will perform minimal SASL negotiation, simulating simple, successful SASL
authentication. We will write specific SASL tests to exercise more complex scenarios.</p>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;" class="grey">
                        <a href="https://cwiki.apache.org/confluence/users/removespacenotification.action?spaceKey=qpid">Stop
watching space</a>
            <span style="padding: 0px 5px;">|</span>
                <a href="https://cwiki.apache.org/confluence/users/editmyemailsettings.action">Change
email notification preferences</a>
</div>
        <a href="https://cwiki.apache.org/confluence/display/qpid/AMQP+1.0+JMS+Client+Requirements+and+Design">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=31823684&revisedVersion=8&originalVersion=7">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/qpid/AMQP+1.0+JMS+Client+Requirements+and+Design?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org


Mime
View raw message