axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aleksander Slominski <as...@cs.indiana.edu>
Subject Re: Timings for processing of small messages
Date Wed, 18 Apr 2001 07:40:43 GMT
Sam Ruby wrote:

> Summary:
>
>    Axis and Xerces 2 provide hope for significantly better response time
>    for small messages then their predecessors.

hi,

i have re-run your tests and added new tests for Xml Pull Parser (modified test sources are
available at
http://www.extreme.indiana.edu/~aslom/echosoap/).

> Parser overhead:
>
>    Alternatives to Xerces 1's implementation of DOM (5533 usec) include
>    JDOM (5224 usec), Crimson (1789 usec), and Xerces 2 (801 usec).  It is
>    worth repeating - these measurements are based on repeatedly parsing
>    small messages in the context of a web server; performance of other
>    scenarios may differ.
>
>    Alternatives to DOM is SAX.  Measurements were 3,050 usec for Xerces 1,
>    1,583 usec for Crimson, and 690 usec for Xerces 2.
>
>    With Xerces 2, a native interface is available.  Timings were 621 usec
>    for the standard configuration, and 531 usec for the non-validating
>    configuration.
>
>    Two non-parser implementations were also created - one was based on
>    strings, the other was based on byte arrays.  The byte array version was
>    faster by 163 usec.  Hopefully this kind of data can be used to identify
>    techniques to target in order to further optimize the soap and axis
>    implementations.

i have got different results (though following your patter)

here are my observations: there is following hierarchy (as measured by number of messages
per second):

#messages per second/#microseconds per message test_name(s)
108/9200 jdom, dom1 (from xerces1.3.0)
150/6500 sax1 (from xerces1.3.0)
255/3900 dom2 (from xerces2)
260/3740 sax2
277/3600 xnin, xnis (internal xerces2 interfaces)
300/3300 char, xpp (Xml Pull Parser 1.1)
330/3030 byte

i think that tests should be re-evaluated as from it one could infer that XPP is  fast as
reading characters directing from input and it seems not
right :-)

i am suspecting that tests are spending too much time doing buffering and memory IO including
socket connection and disconnection that heavily
affects performance - it would be interesting to see tests that uses HTTP keep-alive and allows
for streaming of input into parsers (and not
buffering it). however it requires very careful coding that will not introduce unnecessary
buffering and delays...( it would be also interesting
to do some testing with chunked encoding but this is even more difficult...).

   (...) The xml-soap implementation processed 50.8 messages at the end, whereas
   xml-axis was able to process 93.7 messages per second.  This translates
   to 19,685 microseconds per message for xml-soap, and 10,672 microseconds
   per message for xml-axis.

   Both results are based on the Xerces 1 DOM implementation and Tomcat
   3.3.  The overhead per message for each have been computed at 5,533
   microseconds and 3,548 microseconds respectively - the latter number
   includes wait time for the client.  Subtracting out this fixed cost
   results in an overhead of 10,604 microseconds per message for xml-soap
   and 1,591 microseconds per message for xml-axis.(...)


that seems like extremely good result :-)

i have run also servlet tests (without soap and axis but just dom, sax and xpp)  but the results
here are extremely inconclusive - i was using
tomcat 3.2.1 and during all tests (sax, dome, xpp and xpp2sax which is SAX 1.0 driver on top
of XML Pull Parser) cpu utilization was below 50%!
that means that there are very significant delays somewhere in tomcat IO (i guess) affecting
heavily servlet performance (in tests they are 20x
slower - from 250+ messages per second when standalone to 15-17 messages per second in servlet).

i could not get any good conclusion from tests - calculating overhead it would seem that char
test is _maximum_ performance that parser can
possible achieve (as it is just converting to UNICODE  from byte input stream and it must
be done in any XML parser) so then i could conclude that
XPP _is_ getting this maximum performance but i think it is possible to achieve even better
performance when streaming input into parser...

let me know what you think about it.

thanks,

alek

ps. tests were executed on PII450Mhz/256MB RAM. with Windows 2000 SP1 on java -version:
java version "1.3.1-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1-beta-b15)
Java HotSpot(TM) Client VM (build 1.3.1beta-b15, mixed mode)

and here are actual results:

byte    342.3   2921.0
byte    330.0   3030.0
byte    326.3   3064.0
byte    329.4   3035.0
byte    319.5   3129.0
char    288.4   3467.0
char    297.6   3360.0
char    301.7   3314.0
char    303.3   3297.0
char    304.5   3284.0
sax1    140.2   7132.0
sax1    150.4   6648.0
sax1    153.0   6535.0
sax1    152.6   6553.0
sax1    148.2   6747.0
dom1    96.3    10384.0
dom1    102.3   9775.0
dom1    105.5   9478.0
dom1    105.4   9487.0
dom1    105.3   9496.0
sax2    241.6   4139.0
sax2    267.3   3741.0
sax2    266.9   3746.0
sax2    267.3   3741.0
sax2    267.3   3741.0
dom2    224.3   4458.0
dom2    255.7   3910.0
dom2    256.1   3904.0
dom2    257.8   3878.0
dom2    256.1   3904.0
jdom    83.8    11933.0
jdom    102.7   9737.0
jdom    107.7   9285.0
jdom    108.2   9242.0
jdom    104.5   9569.0
xnin    253.7   3941.0
xnin    278.0   3597.0
xnin    277.5   3603.0
xnin    278.6   3589.0
xnin    277.5   3603.0
xnis    248.6   4022.0
xnis    272.7   3667.0
xnis    274.1   3648.0
xnis    272.6   3668.0
xnis    273.6   3654.0
xpp     290.0   3448.0
xpp     299.0   3344.0
xpp     301.7   3314.0
xpp     300.8   3324.0
xpp     301.4   3317.0
echo/xpp        17.7    56497.0
echo/xpp        16.9    59171.0
echo/xpp        17.7    56497.0
echo/xpp        17.7    56497.0
echo/xpp        17.8    56179.0
echo/xpp2sax    17.7    56497.0
echo/xpp2sax    17.6    56818.0
echo/xpp2sax    17.6    56818.0
echo/xpp2sax    17.6    56818.0
echo/xpp2sax    17.8    56179.0
echo/sax        16.6    60240.0
echo/sax        16.3    61349.0
echo/sax        16.6    60240.0
echo/sax        16.3    61349.0
echo/sax        16.6    60240.0
echo/dom        15.2    65789.0
echo/dom        15.2    65789.0
echo/dom        14.6    68493.0
echo/dom        15.0    66666.0
echo/dom        15.0    66666.0

--
Aleksander Slominski, LH 316, IU, http://www.extreme.indiana.edu/~aslom
As I look afar I see neither cherry Nor tinted leaves Just a modest hut
on the coast In the dusk of Autumn nightfall - Fujiwara no Teika(1162-1241)



Mime
View raw message