axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aleksander Slominski <as...@cs.indiana.edu>
Subject Re: more on AXIS-Java/C++ server side performance
Date Mon, 14 Jun 2004 16:34:49 GMT
Susantha Kumara wrote:

>Hi Alek
>
>You are doing a great work...!
>
>Which parser did you use ?. Xerces or Expat. 
>
expat.

>Last time we did
>performance tests we found that Axis C++ is faster with Expat. 3 times
>faster with Expat than Xerces.
>  
>
ha tis ehat i expected as expat is supposed ot be the fastest XML parser 
on C.

i will post more results and a graph soon.

thanks,

alek

>>-----Original Message-----
>>From: Aleksander Slominski [mailto:aslom@cs.indiana.edu]
>>Sent: Saturday, June 12, 2004 5:34 PM
>>To: Apache AXIS C Developers List; 'axis-dev@ws.apache.org'
>>Subject: more on AXIS-Java/C++ server side performance
>>
>>Davanum Srinivas wrote:
>>
>>    
>>
>>>Alek,
>>>
>>>Where is the sample code for your XSOAP server-side code? does it
>>>create a java object from the soap request?
>>>
>>>      
>>>
>>hi dims,
>>
>>first let me emphasize that i do not look on this benchmark as only to
>>indicate performance but also a tool
>>
>>now that i said that i think i can demonstrate usefulness of such
>>    
>>
>tests:
>  
>
>>to my big surprise after running tests i can see that AXIS-C++ is two
>>times slower than AXIS-Java (in this sense the test work to help
>>    
>>
>detect
>  
>
>>possible huge performance bottlenecks may ultimately lead to much
>>    
>>
>better
>  
>
>>implementation).
>>
>>anyway this is really strange - i would expect C++ to be faster :)
>>
>>below is everything you need to reproduce test results yourself and
>>maybe somebody will have an idea what is going on with AXIS-C++ ...
>>
>>the test driver (client side) is still not fully finished but here is
>>working version:
>>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/driver/
>>
>>after downloading do source classpath.sh (or classpath.bat on windows)
>>to set CLASSPATH
>>
>>run self test:
>>java -Dserver.name=SELF_TEST soap_bench.BenchDriver
>>    
>>
>http://localhost:3434
>  
>
>>and here is full description of command line:
>>
>>java -Dserver.name=SERVICE_NAME soap_bench.Bench URL total
>>{rsea}{bdisva} [arraySize][,arraySize][,...]
>>
>>parameters in order:
>>1. URL of service
>>2. total number of elements to send (default 10K)
>>3. specification of what to send : two letters
>>     [rse] means receive, send, or echo (a == all)
>>     [bdisva] means base64, double, int, string, void (only applies to
>>echo), a == all methods;
>>4. list of arraySizes (optional for void) - default to 10
>>and you can set test metadata such as  -Dmachine.name=...
>>-Dserver.name=...  (it is included in tabulated results good for
>>spreadsheet input and to draw graphs)
>>
>>those are optional system properties:
>>-Dsmoke_test - will show you what messages were received and check
>>    
>>
>that
>  
>
>>all works fine
>>-DAXISCPP - will add SOAPAction header and xsi:type for BASE64 (so
>>AXIS-C++ is happy)
>>-Dlog=trace.xsul.http.client:ALL  -will show all messages and HTTP
>>headers (good for debugging)
>>-Dlog=:ALL  - if you are doing desperate debugging
>>
>>use smoke test to get code to work with driver - it will just send 3
>>messages and will print what it received - combine it with -Dlog=...
>>    
>>
>to
>  
>
>>get more fine grained information on what is happening and if/when
>>driver is failing.
>>
>>the way i would do to get test driver to work with a SOAP service is
>>    
>>
>to
>  
>
>>it in multiple esteps and to get first to work different echo methods
>>
>>a) echoVoid
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>ev
>  
>
>>10,100
>>
>>echoInts
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>ei
>  
>
>>10,100
>>
>>echoDoubles
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>ed
>  
>
>>10,100
>>
>>echoStrings
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>es
>  
>
>>10,100
>>
>>echoBytes (as Base64)
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>eb
>  
>
>>10,100
>>
>>b) all echo methods
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>ea
>  
>
>>10,100
>>
>>c) all methods
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient  <URL> 200
>>    
>>
>aa
>  
>
>>10,100
>>
>>and then run real tests
>>
>>----------------------
>>For AXIS-Java
>>
>>get code from
>>    
>>
>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_java/
>  
>
>>and deploy benchmark service in tomcat - to do it just modify
>>build.properties and do ant all
>>
>>run test
>>java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=AXIS_1_2_CVS
>>soap_bench.BenchClient http://server:8080/axis/services/Benchmark1
>>200000 aa 10,1000,10000
>>
>>----------------------
>>For AXIS-C++
>>
>>get code from
>>    
>>
>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_cpp/
>  
>
>>copy soap_bench/ to samples/server (and client respectively)
>>modify top level Makefile.am to include soap_bench (see example)
>>rebuilt .so library
>>add to list of deployed services (see deploy.wsdd in soap_bench)
>>restart apache server
>>
>>run test (make sure -DAXISCPP parameter is included!!!!!)
>>java -DAXISCPP -Dmachine.name=YOUR_MACHINE_NAME
>>-Dserver.name=AXISCPP_1_0_CVS
>>soap_bench.BenchClient http://server:8080/axis/Benchmark1 200000 ea
>>10,1000,10000
>>
>>NOTE: 'aa' will not work as all send tests do not work (and i do not
>>know why). instead try ea (echo tests) and sa (send tests) - if you
>>    
>>
>want
>  
>
>>to see what is not working use sa and enable logging (-Dlog=:ALL)
>>
>>--------------------------
>>For XSOAP4
>>
>>start service:  java soap_bench.BenchServer 8077
>>
>>run test driver:
>>java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=XSOAP4
>>soap_bench.BenchClient http://server:8077 200000 aa 10,1000,10000
>>
>>----------------------
>>
>>comments are welcome especially concerning performance differences
>>between Java and C++ and how to get send tests to work with AXIS
>>    
>>
>C++...
>  
>
>>thanks,
>>
>>alek
>>
>>    
>>
>>>On Thu, 03 Jun 2004 12:58:41 -0500, Aleksander Slominski
>>><aslom@cs.indiana.edu> wrote:
>>>
>>>
>>>      
>>>
>>>>Davanum Srinivas wrote:
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>You need to turn streaming on the client-side as well. see
>>>>>samples/perf in latest cvs.
>>>>>
>>>>>
>>>>>          
>>>>>
>>binding._setProperty(org.apache.axis.client.Call.STREAMING_PROPERTY,
>>    
>>
>>>>>Boolean.TRUE);
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>i am interested in testing server side performance (as server side
>>>>        
>>>>
>is
>  
>
>>>>typically the bottleneck) so i use the same client to access all
>>>>services to measure only factors that are effecting server side
>>>>performance (CPU, memory footprint, scalability).
>>>>
>>>>hopefully soon i will have AXIS-C++ numbers - i am really close now
>>>>        
>>>>
>to
>  
>
>>>>get them after few weeks of struggling with C++ (Java configuration
>>>>        
>>>>
>and
>  
>
>>>>build tools are much easier even when dealing with JAR hell  ...)
>>>>
>>>>so it should be interesting to see how AXIS-Java and AXIS-C++
>>>>        
>>>>
>compare.
>  
>
>>>>however i am also *very* interested in very long running web
>>>>        
>>>>
>services so
>  
>
>>>>Peter post about _huge_ memory leaks that are easily reproducible
>>>>        
>>>>
>looked
>  
>
>>>>very interesting  to me :) is it a property of data graph he send or
>>>>        
>>>>
>is
>  
>
>>>>it just a bug ...
>>>>
>>>>thanks,
>>>>
>>>>alek
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>-- dims
>>>>>
>>>>>On Thu, 03 Jun 2004 12:27:18 -0500, Aleksander Slominski
>>>>><aslom@cs.indiana.edu> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>>>Peter,
>>>>>>
>>>>>>please post more details how to reproduce the memory leak and i
>>>>>>            
>>>>>>
>will
>  
>
>>add
>>    
>>
>>>>>>to what i have for WS/SOAP performance tests at:
>>>>>>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/
>>>>>>
>>>>>>thanks,
>>>>>>
>>>>>>alek
>>>>>>
>>>>>>Peter Molettiere wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>>>>>We're currently looking into a massive memory leak in axis, that
>>>>>>>              
>>>>>>>
>no
>  
>
>>>>>>>one  here seems willing to admit. We're close to having a minimal
>>>>>>>              
>>>>>>>
>>test
>>    
>>
>>>>>>>case  which demonstrates the leak we're seeing. I don't think
>>>>>>>              
>>>>>>>
>it's
>  
>
>>the
>>    
>>
>>>>>>>same  one you're seeing however, as our leak happens 1) when the
>>>>>>>server is  sitting doing nothing (but only after axis has been
>>>>>>>loaded), and 2)  every time we serialize a large message.  The
>>>>>>>suggestions on the list  about switching to using attachments
>>>>>>>              
>>>>>>>
>doesn't
>  
>
>>>>>>>fix the leak sadly. And  setting a larger heap size only delays
>>>>>>>              
>>>>>>>
>the
>  
>
>>>>>>>inevitable Out Of Memory  Exception.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>>>From our testing, it looks like it takes on the order of 700MB
to
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>>>>>serialize a 30MB java object graph into a 3MB SOAP stream.
>>>>>>>
>>>>>>>I've posted about this before, but no one has responded.
>>>>>>>              
>>>>>>>
>Hopefully
>  
>
>>>>>>>when  we post the test case someone will actually take a look
at
>>>>>>>              
>>>>>>>
>it.
>  
>
>>>>>>>There  are other threads with test cases which currently have
>>>>>>>              
>>>>>>>
>"we're
>  
>
>>>>>>>looking  at this now" as the only response, but as they are all
>>>>>>>              
>>>>>>>
>>pretty
>>    
>>
>>>>>>>old  threads, my guess is that this problem either isn't believed
>>>>>>>              
>>>>>>>
>or
>  
>
>>>>>>>is  being swept under the carpet. Maybe we're the only people
>>>>>>>              
>>>>>>>
>using
>  
>
>>>>>>>axis  for large messages.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>--
>>>>>>>
>>>>>>>On Jun 3, 2004, at 9:38 AM, jira@apache.org wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>>>The following comment has been added to this issue:
>>>>>>>>
>>>>>>>>   Author: Jennifer Jackson
>>>>>>>>  Created: Thu, 3 Jun 2004 9:37 AM
>>>>>>>>     Body:
>>>>>>>>Has anyone looked into this?  I haven't used Axis because
of
>>>>>>>>                
>>>>>>>>
>this
>  
>
>>>>>>>>problem and doing a quick search it seems others have come
>>>>>>>>                
>>>>>>>>
>across
>  
>
>>>>>>>>memory leaks also.  I narrowed it down to the Call object
not
>>>>>>>>cleaning  out its member vars (the whole Deserialization
>>>>>>>>                
>>>>>>>>
>process).
>  
>
>>>>>>>>If you do  that the only thing stuck in memory is the call
>>>>>>>>                
>>>>>>>>
>object
>  
>
>>>>>>>>since its part  of the ThreadLocal object.
>>>>>>>>                
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>              
>>>>>>>
>--
>  
>
>>-
>>    
>>
>>>>>>>>View this comment:
>>>>>>>> http://issues.apache.org/jira/browse/AXIS-935?
>>>>>>>>page=comments#action_35890
>>>>>>>>
>>>>>>>>                
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>              
>>>>>>>
>--
>  
>
>>-
>>    
>>
>>>>>>>>View the issue:
>>>>>>>>http://issues.apache.org/jira/browse/AXIS-935
>>>>>>>>
>>>>>>>>Here is an overview of the issue:
>>>>>>>>                
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>              
>>>>>>>
>--
>  
>
>>-
>>    
>>
>>>>>>>>      Key: AXIS-935
>>>>>>>>  Summary: Using ThreadLocal Call objects in a servlet memory
>>>>>>>>                
>>>>>>>>
>leak
>  
>
>>>>>>>>both on client and server side
>>>>>>>>     Type: Bug
>>>>>>>>
>>>>>>>>   Status: Open
>>>>>>>>
>>>>>>>>  Project: Axis
>>>>>>>>Components:
>>>>>>>>           Basic Architecture
>>>>>>>> Versions:
>>>>>>>>           1.1rc2
>>>>>>>>
>>>>>>>> Assignee: Axis Developers Mailing List
>>>>>>>> Reporter: Jennifer Jackson
>>>>>>>>
>>>>>>>>  Created: Mon, 30 Jun 2003 12:12 PM
>>>>>>>>  Updated: Thu, 3 Jun 2004 9:37 AM
>>>>>>>>Environment: Operating System: All
>>>>>>>>Platform: All
>>>>>>>>
>>>>>>>>Description:
>>>>>>>>I discovered using Axis in tomcat causes a memory leak.
>>>>>>>>
>>>>>>>>It seems using threadlocal in the classes causes a memory
leak
>>>>>>>>                
>>>>>>>>
>for
>  
>
>>>>>>>>each Call
>>>>>>>>that is created.  Threadlocal sticks around until the calling
>>>>>>>>                
>>>>>>>>
>thread
>  
>
>>>>>>>>dies, and
>>>>>>>>since tomcat keeps threads alive to answer new calls, the
>>>>>>>>threadlocal  data
>>>>>>>>never goes away.  The Call object gets stuck in the threadlocal
>>>>>>>>data,  which
>>>>>>>>contains the entire soap response and all objects.  This is
a
>>>>>>>>                
>>>>>>>>
>BIG
>  
>
>>>>>>>>memory leak.
>>>>>>>>I initially cleared it by forcing all Call objects to clean
>>>>>>>>                
>>>>>>>>
>their
>  
>
>>>>>>>>member vars
>>>>>>>>after I was done with a call, then I realized the source was
>>>>>>>>                
>>>>>>>>
>from
>  
>
>>>>>>>>the  thread
>>>>>>>>local class.  Here is some info I found about threadlocal:
>>>>>>>>
>>>>>>>>This is stated in the Sun Javadocs for ThreadLocal:
>>>>>>>>
>>>>>>>>  Each thread holds an implicit reference to its copy of a
>>>>>>>>  ThreadLocal as long as the thread is alive and the
>>>>>>>>  ThreadLocal object is accessible; after a thread goes
>>>>>>>>  away, all of its copies of ThreadLocal variables are
>>>>>>>>  subject to garbage collection (unless other references
>>>>>>>>  to these copies exist).
>>>>>>>>
>>>>>>>>So, this means that ANY APPLICATION that uses PreparedStatements
>>>>>>>>                
>>>>>>>>
>in
>  
>
>>>>>>>>a  thread
>>>>>>>>that 1) either does a lot of PreparedStatements or 2) never
dies
>>>>>>>>(i.e., a
>>>>>>>>main thread) will ALWAYS eventually have an out of memory
error.
>>>>>>>>Simply put,
>>>>>>>>this is a MEMORY LEAK. I imagine that the leak is very small,
>>>>>>>>                
>>>>>>>>
>the
>  
>
>>>>>>>>ThreadLocal object only contains one member variable, maybe
64
>>>>>>>>                
>>>>>>>>
>bytes
>  
>
>>>>>>>>or less
>>>>>>>>(depending on the VM implementation). So, our 60,000
>>>>>>>>PreparedStatements of 2
>>>>>>>>ThreadLocals each times 64 bytes (my wild guess) is 7.5MB.
>>>>>>>>
>>>>>>>>Ideas?  I've never used threadlocal myself so this is new
to me.
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>              
>>>>>>>
>--
>  
>
>>-
>>    
>>
>>>>>>>>JIRA INFORMATION:
>>>>>>>>This message is automatically generated by JIRA.
>>>>>>>>
>>>>>>>>If you think it was sent incorrectly contact one of the
>>>>>>>>                
>>>>>>>>
>>administrators:
>>    
>>
>>>>>>>> http://issues.apache.org/jira/secure/Administrators.jspa
>>>>>>>>
>>>>>>>>If you want more information on JIRA, or have a bug to report
>>>>>>>>                
>>>>>>>>
>see:
>  
>
>>>>>>>> http://www.atlassian.com/software/jira
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>--
>>>>>>The best way to predict the future is to invent it - Alan Kay
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>>--
>>>>The best way to predict the future is to invent it - Alan Kay
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>>--
>>The best way to predict the future is to invent it - Alan Kay
>>    
>>
>
>
>  
>


-- 
The best way to predict the future is to invent it - Alan Kay


Mime
View raw message