qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Broadstone <mbroa...@gmail.com>
Subject Re: Qpid Interop Test Suite
Date Mon, 28 Nov 2016 16:14:15 GMT
On Mon, Nov 28, 2016 at 11:02 AM, Kim van der Riet
<kim.vdriet@redhat.com> wrote:
> I have been working for a while now on a set of interoperability tests for
> our AMQP 1.0 clients.  The idea is to have an installable set of tests that
> can be easily run against any running broker which combines our clients so
> that they send and receive messages from each other and test that the
> received messages match the sent ones.
>
> Download: https://git-wip-us.apache.org/repos/asf/qpid-interop-test.git
> JIRA: https://issues.apache.org/jira/browse/QPIDIT/
>
> These tests are end-to-end compatibility tests. Although the idea is to use
> them to test primarily the compatibility of our clients with each other,
> they also indirectly test that the broker does its part in handling messages
> containing a complete set of AMQP and JMS message types and content.  The
> tests don't look at what is happening on the wire at all, they only send
> message content and compare it to received message content.
>
> The currently available tests are:
> * AMQP type tests - send data in each of the official AMQP 1.0 simple and
> compound types;
> * AMQP large content tests - send and receive messages containing bodies and
> headers with large amounts of data (up to 100MB).
> * JMS messages test - sends and receives messages from all clients in a
> Qpid-JMS-compatible format.
> * JMS headers and properties test - sends and receives all the
> client-settable JMS headers and all possible JMS property types in various
> combinations.
>
> The current clients are:
> * Proton C++
> * Proton Python
> * Rhea (Javascript)
> * Qpid JMS (Only JMS tests at present)
>
> The tests are designed to be reasonably modular so that it is easy to add
> both tests and client shims.
>
> The tests run as follows:
>
> * A Python test control program is run which coordinates the tests and test
> data (ie message/header/data content for each test)
> * As the test runs, the control program sends a structured set of parameters
> to a sender "shim" (a simple client program) whose job it is to interpret
> the data sent to it as message content and send those messages. Similarly,
> the control program sends a request to receive messages to a receiver "shim"
> whose job it is to receive messages and send the received data back to the
> control program
> * The control program then compares the sent data to the received data and
> passes or fails the test accordingly.
>
> I have just made the tests installable which should make it easy to obtain,
> build and run.  The tests need Qpid Proton to be installed and built, also
> Rhea and Proton Python. The docs are still somewhat spotty, but I'll be
> happy to take feedback and answer questions on the list.
>
> Future work includes the following:
> * Get some visibility on the Apache Qpid web pages
> * AMQP features test - tests the features of AMQP which affect or rely on
> interoperability between two clients
> * Transactions testing - test that two clients that share in a transaction
> can interoperate
> * Add some additional client shims as the need arises (Go, PHP, Perl
> anyone?)
> * Fill in a few holes in the test capabilities (like compound AMQP types
> using more than just strings).
> * How about getting a version of this running on Apache CI infrastructure?
>
> I would like to encourage the community to take a look at this test suite
> and give it a try. I would be interested to hear any feedback, and I'll be
> happy to answer any questions about it you may have. Of course, you are also
> welcome to file issues on the project JIRA (the project uses the QPIDIT key
> on the Apache JIRA.
>
> Once I receive some feedback, I hope to release a snapshot of the current
> work, followed by an full release.
>
> Thanks for your help,
> Kim
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>


Hi Kim,

I'm the maintainer of a node.js AMQP 1.0 client
(https://github.com/noodlefrenzy/node-amqp10) and am highly interested
in finding a way to incorporate our client into your tests if at all
possible. We currently run a limited set of integration tests against
the latest released qpidd in TravisCI, however I gather this would be
far more comprehensive.

To that end, a few questions:

  * is it out of scope to include an external library in these tests?
/ would you be interested in such work
  * it seems like this is simply a matter of providing a shim similar
to the one provided for rhea, is there anything else I might be
missing there?
  * I'm sure that we have gaps in our implementation in terms of full
coverage, if the answer to the first question is (no, then yes), what
is your timeline for a release of the test suite?

Regards,
Matt Broadstone

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message