axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aleksander Slominski <as...@cs.indiana.edu>
Subject more on AXIS-Java/C++ server side performance
Date Sat, 12 Jun 2004 11:34:17 GMT
Davanum Srinivas wrote:

>Alek,
>
>Where is the sample code for your XSOAP server-side code? does it
>create a java object from the soap request?  
>
hi dims,

first let me emphasize that i do not look on this benchmark as only to 
indicate performance but also a tool

now that i said that i think i can demonstrate usefulness of such tests: 
to my big surprise after running tests i can see that AXIS-C++ is two 
times slower than AXIS-Java (in this sense the test work to help detect 
possible huge performance bottlenecks may ultimately lead to much better 
implementation).

anyway this is really strange - i would expect C++ to be faster :)

below is everything you need to reproduce test results yourself and 
maybe somebody will have an idea what is going on with AXIS-C++ ...

the test driver (client side) is still not fully finished but here is 
working version:
http://www.extreme.indiana.edu/~aslom/bnp/wsperf/driver/

after downloading do source classpath.sh (or classpath.bat on windows) 
to set CLASSPATH

run self test:
java -Dserver.name=SELF_TEST soap_bench.BenchDriver http://localhost:3434

and here is full description of command line:

java -Dserver.name=SERVICE_NAME soap_bench.Bench URL total 
{rsea}{bdisva} [arraySize][,arraySize][,...]

parameters in order:
1. URL of service
2. total number of elements to send (default 10K)
3. specification of what to send : two letters
     [rse] means receive, send, or echo (a == all)
     [bdisva] means base64, double, int, string, void (only applies to 
echo), a == all methods;
4. list of arraySizes (optional for void) - default to 10
and you can set test metadata such as  -Dmachine.name=... 
-Dserver.name=...  (it is included in tabulated results good for 
spreadsheet input and to draw graphs)

those are optional system properties:
-Dsmoke_test - will show you what messages were received and check that 
all works fine
-DAXISCPP - will add SOAPAction header and xsi:type for BASE64 (so 
AXIS-C++ is happy)
-Dlog=trace.xsul.http.client:ALL  -will show all messages and HTTP 
headers (good for debugging)
-Dlog=:ALL  - if you are doing desperate debugging

use smoke test to get code to work with driver - it will just send 3 
messages and will print what it received - combine it with -Dlog=... to 
get more fine grained information on what is happening and if/when 
driver is failing.

the way i would do to get test driver to work with a SOAP service is to 
it in multiple esteps and to get first to work different echo methods

a) echoVoid

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 ev 
10,100

echoInts

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 ei 
10,100

echoDoubles

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 ed 
10,100

echoStrings

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 es 
10,100

echoBytes (as Base64)

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 eb 
10,100

b) all echo methods

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 ea 
10,100

c) all methods

java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200 aa 
10,100

and then run real tests

----------------------
For AXIS-Java

get code from http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_java/
and deploy benchmark service in tomcat - to do it just modify 
build.properties and do ant all

run test
java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=AXIS_1_2_CVS
soap_bench.BenchClient http://server:8080/axis/services/Benchmark1 
200000 aa 10,1000,10000

----------------------
For AXIS-C++

get code from http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_cpp/
copy soap_bench/ to samples/server (and client respectively)
modify top level Makefile.am to include soap_bench (see example)
rebuilt .so library
add to list of deployed services (see deploy.wsdd in soap_bench)
restart apache server

run test (make sure -DAXISCPP parameter is included!!!!!)
java -DAXISCPP -Dmachine.name=YOUR_MACHINE_NAME 
-Dserver.name=AXISCPP_1_0_CVS
soap_bench.BenchClient http://server:8080/axis/Benchmark1 200000 ea 
10,1000,10000

NOTE: 'aa' will not work as all send tests do not work (and i do not 
know why). instead try ea (echo tests) and sa (send tests) - if you want 
to see what is not working use sa and enable logging (-Dlog=:ALL)

--------------------------
For XSOAP4

start service:  java soap_bench.BenchServer 8077

run test driver:
java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=XSOAP4
soap_bench.BenchClient http://server:8077 200000 aa 10,1000,10000

----------------------

comments are welcome especially concerning performance differences 
between Java and C++ and how to get send tests to work with AXIS C++...

thanks,

alek

>On Thu, 03 Jun 2004 12:58:41 -0500, Aleksander Slominski
><aslom@cs.indiana.edu> wrote:
>  
>
>>Davanum Srinivas wrote:
>>
>>    
>>
>>>You need to turn streaming on the client-side as well. see
>>>samples/perf in latest cvs.
>>>
>>>       binding._setProperty(org.apache.axis.client.Call.STREAMING_PROPERTY,
>>>Boolean.TRUE);
>>>
>>>
>>>      
>>>
>>i am interested in testing server side performance (as server side is
>>typically the bottleneck) so i use the same client to access all
>>services to measure only factors that are effecting server side
>>performance (CPU, memory footprint, scalability).
>>
>>hopefully soon i will have AXIS-C++ numbers - i am really close now to
>>get them after few weeks of struggling with C++ (Java configuration and
>>build tools are much easier even when dealing with JAR hell  ...)
>>
>>so it should be interesting to see how AXIS-Java and AXIS-C++ compare.
>>
>>however i am also *very* interested in very long running web services so
>>Peter post about _huge_ memory leaks that are easily reproducible looked
>>very interesting  to me :) is it a property of data graph he send or is
>>it just a bug ...
>>
>>thanks,
>>
>>alek
>>
>>    
>>
>>>-- dims
>>>
>>>On Thu, 03 Jun 2004 12:27:18 -0500, Aleksander Slominski
>>><aslom@cs.indiana.edu> wrote:
>>>
>>>
>>>      
>>>
>>>>Peter,
>>>>
>>>>please post more details how to reproduce the memory leak and i will add
>>>>to what i have for WS/SOAP performance tests at:
>>>>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/
>>>>
>>>>thanks,
>>>>
>>>>alek
>>>>
>>>>Peter Molettiere wrote:
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>We're currently looking into a massive memory leak in axis, that no
>>>>>one  here seems willing to admit. We're close to having a minimal test
>>>>>case  which demonstrates the leak we're seeing. I don't think it's the
>>>>>same  one you're seeing however, as our leak happens 1) when the
>>>>>server is  sitting doing nothing (but only after axis has been
>>>>>loaded), and 2)  every time we serialize a large message.  The
>>>>>suggestions on the list  about switching to using attachments doesn't
>>>>>fix the leak sadly. And  setting a larger heap size only delays the
>>>>>inevitable Out Of Memory  Exception.
>>>>>
>>>>>          
>>>>>
>>>>>>From our testing, it looks like it takes on the order of 700MB to
>>>>        
>>>>
>>>>>serialize a 30MB java object graph into a 3MB SOAP stream.
>>>>>
>>>>>I've posted about this before, but no one has responded. Hopefully
>>>>>when  we post the test case someone will actually take a look at it.
>>>>>There  are other threads with test cases which currently have "we're
>>>>>looking  at this now" as the only response, but as they are all pretty
>>>>>old  threads, my guess is that this problem either isn't believed or
>>>>>is  being swept under the carpet. Maybe we're the only people using
>>>>>axis  for large messages.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>--
>>>>>
>>>>>On Jun 3, 2004, at 9:38 AM, jira@apache.org wrote:
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>>>The following comment has been added to this issue:
>>>>>>
>>>>>>    Author: Jennifer Jackson
>>>>>>   Created: Thu, 3 Jun 2004 9:37 AM
>>>>>>      Body:
>>>>>>Has anyone looked into this?  I haven't used Axis because of this
>>>>>>problem and doing a quick search it seems others have come across
>>>>>>memory leaks also.  I narrowed it down to the Call object not
>>>>>>cleaning  out its member vars (the whole Deserialization process).
>>>>>>If you do  that the only thing stuck in memory is the call object
>>>>>>since its part  of the ThreadLocal object.
>>>>>>---------------------------------------------------------------------
>>>>>>View this comment:
>>>>>>  http://issues.apache.org/jira/browse/AXIS-935?
>>>>>>page=comments#action_35890
>>>>>>
>>>>>>---------------------------------------------------------------------
>>>>>>View the issue:
>>>>>> http://issues.apache.org/jira/browse/AXIS-935
>>>>>>
>>>>>>Here is an overview of the issue:
>>>>>>---------------------------------------------------------------------
>>>>>>       Key: AXIS-935
>>>>>>   Summary: Using ThreadLocal Call objects in a servlet memory leak
>>>>>>both on client and server side
>>>>>>      Type: Bug
>>>>>>
>>>>>>    Status: Open
>>>>>>
>>>>>>   Project: Axis
>>>>>>Components:
>>>>>>            Basic Architecture
>>>>>>  Versions:
>>>>>>            1.1rc2
>>>>>>
>>>>>>  Assignee: Axis Developers Mailing List
>>>>>>  Reporter: Jennifer Jackson
>>>>>>
>>>>>>   Created: Mon, 30 Jun 2003 12:12 PM
>>>>>>   Updated: Thu, 3 Jun 2004 9:37 AM
>>>>>>Environment: Operating System: All
>>>>>>Platform: All
>>>>>>
>>>>>>Description:
>>>>>>I discovered using Axis in tomcat causes a memory leak.
>>>>>>
>>>>>>It seems using threadlocal in the classes causes a memory leak for
>>>>>>each Call
>>>>>>that is created.  Threadlocal sticks around until the calling thread
>>>>>>dies, and
>>>>>>since tomcat keeps threads alive to answer new calls, the
>>>>>>threadlocal  data
>>>>>>never goes away.  The Call object gets stuck in the threadlocal
>>>>>>data,  which
>>>>>>contains the entire soap response and all objects.  This is a BIG
>>>>>>memory leak.
>>>>>>I initially cleared it by forcing all Call objects to clean their
>>>>>>member vars
>>>>>>after I was done with a call, then I realized the source was from
>>>>>>the  thread
>>>>>>local class.  Here is some info I found about threadlocal:
>>>>>>
>>>>>>This is stated in the Sun Javadocs for ThreadLocal:
>>>>>>
>>>>>>   Each thread holds an implicit reference to its copy of a
>>>>>>   ThreadLocal as long as the thread is alive and the
>>>>>>   ThreadLocal object is accessible; after a thread goes
>>>>>>   away, all of its copies of ThreadLocal variables are
>>>>>>   subject to garbage collection (unless other references
>>>>>>   to these copies exist).
>>>>>>
>>>>>>So, this means that ANY APPLICATION that uses PreparedStatements in
>>>>>>a  thread
>>>>>>that 1) either does a lot of PreparedStatements or 2) never dies
>>>>>>(i.e., a
>>>>>>main thread) will ALWAYS eventually have an out of memory error.
>>>>>>Simply put,
>>>>>>this is a MEMORY LEAK. I imagine that the leak is very small, the
>>>>>>ThreadLocal object only contains one member variable, maybe 64 bytes
>>>>>>or less
>>>>>>(depending on the VM implementation). So, our 60,000
>>>>>>PreparedStatements of 2
>>>>>>ThreadLocals each times 64 bytes (my wild guess) is 7.5MB.
>>>>>>
>>>>>>Ideas?  I've never used threadlocal myself so this is new to me.
>>>>>>
>>>>>>
>>>>>>---------------------------------------------------------------------
>>>>>>JIRA INFORMATION:
>>>>>>This message is automatically generated by JIRA.
>>>>>>
>>>>>>If you think it was sent incorrectly contact one of the administrators:
>>>>>>  http://issues.apache.org/jira/secure/Administrators.jspa
>>>>>>
>>>>>>If you want more information on JIRA, or have a bug to report see:
>>>>>>  http://www.atlassian.com/software/jira
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>>>          
>>>>>
>>>>--
>>>>The best way to predict the future is to invent it - Alan Kay
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>>--
>>The best way to predict the future is to invent it - Alan Kay
>>
>>
>>    
>>


-- 
The best way to predict the future is to invent it - Alan Kay


Mime
View raw message