axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Ruby" <ru...@us.ibm.com>
Subject Timings for processing of small messages
Date Wed, 18 Apr 2001 02:14:58 GMT
Summary:

   Axis and Xerces 2 provide hope for significantly better response time
   for small messages then their predecessors.

Disclaimers:

   Processing of small messages is not the only scenario, but I would argue
   it is an important one.  Particularly from the perspective of measuring
   overhead.  This test is also designed to measure latency (time to send
   and receive a request) under rather ideal conditions (one client,
   request is completely processed before the next is issued).  Other
   measurements are important, like throughput (number of messages a server
   can process under heavy loads).

   These disclaimers aside, this is just the first scenario I have analyzed
   in any detail.  It is likely to identify areas of fixed overhead, and
   any improvement made here will likely benefit all scenarios and will
   improve throughput too.  I designed these tests unaware of what the
   results would be.  The tests were not designed to produce a specific
   result.  I encourage others to reproduce the results.  All sources are
   attached.

Methodology.

   I sent a rather small but syntactically correct soap message, and
   generated an appropriate reply.  The reply contains data found in the
   request, so parsing is required.  In the soap and axis cases,
   conversions to and from floating point were done, in all others the data
   was simply copied as is.  In each case, the results were verified using
   the TcpTunnelGui tool.  The number of request/response pairs were
   counted in periods of eleven seconds each, with the first second
   ignored.  Each test was repeated five times, with the best time taken.
   As a general rule, timings improved each time, a phenomenon I attribute
   to hotspot.  The messages were designed so that they could be processed
   by MS dot net beta 1, the Apache xml-soap, and the Apache xml-axis
   implementations.  All tests were performed with the latest versions of
   each of the products compiled from source from CVS within the last 24
   hours.

The standard message was:

   <?xml version="1.0"?>
   <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsi="
   http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema">
   <soap:Body>
   <echoFloat xmlns="http://tempuri.org/">
   <input xsi:type="xsd:float">3.14159</input>
   </echoFloat>
   </soap:Body>
   </soap:Envelope>

With a response of:

   <?xml version="1.0"?>
   <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsi="
   http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema">
   <soap:Body>
   <echoFloatResult xmlns="http://tempuri.org/">
   <result>3.14159</result>
   </echoFloatResult>
   </soap:Body>
   </soap:Envelope>

Overall results:

   The xml-soap implementation processed 50.8 messages at the end, whereas
   xml-axis was able to process 93.7 messages per second.  This translates
   to 19,685 microseconds per message for xml-soap, and 10,672 microseconds
   per message for xml-axis.

   Both results are based on the Xerces 1 DOM implementation and Tomcat
   3.3.  The overhead per message for each have been computed at 5,533
   microseconds and 3,548 microseconds respectively - the latter number
   includes wait time for the client.  Subtracting out this fixed cost
   results in an overhead of 10,604 microseconds per message for xml-soap
   and 1,591 microseconds per message for xml-axis.

Parser overhead:

   Alternatives to Xerces 1's implementation of DOM (5533 usec) include
   JDOM (5224 usec), Crimson (1789 usec), and Xerces 2 (801 usec).  It is
   worth repeating - these measurements are based on repeatedly parsing
   small messages in the context of a web server; performance of other
   scenarios may differ.

   Alternatives to DOM is SAX.  Measurements were 3,050 usec for Xerces 1,
   1,583 usec for Crimson, and 690 usec for Xerces 2.

   With Xerces 2, a native interface is available.  Timings were 621 usec
   for the standard configuration, and 531 usec for the non-validating
   configuration.

   Two non-parser implementations were also created - one was based on
   strings, the other was based on byte arrays.  The byte array version was
   faster by 163 usec.  Hopefully this kind of data can be used to identify
   techniques to target in order to further optimize the soap and axis
   implementations.

Configuration:

   550 Mhz Pentium III
   196M RAM
   Sun JDK 1.3 w/hotspot
   Win2K server

- Sam Ruby

(See attached file: echosoap.zip)
Mime
View raw message