axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davanum Srinivas <dava...@gmail.com>
Subject Re: AXIS-1423 : Peter's test case for performance
Date Mon, 19 Jul 2004 16:57:56 GMT
Nishant,

Switch ByteArray to use Commons' BOAS. Can you please try latest CVS
and help us tune the impl? Are u willing to donate your BOAS to Apache
for inclusion with Axis?

thanks,
dims

On Sat, 17 Jul 2004 08:39:05 -0400, Davanum Srinivas <davanum@gmail.com> wrote:
> Nishant, Peter,
> 
> Am willing to accept any patches against ByteArray incorporating the
> concepts in your BOAS or commons BOAS. Since you have dived deep into
> the code, can you please submit a patch? (please keep the file backup
> code in case of REALLY big messages and please keep the existing flags
> if possible.)
> 
> As u said earlier, it is not the cause of the OOM of peter. and am
> still unable to recreate it. I used JMeter for client-side and ran the
> server on OptimizeIT. To set the stage i invoked the client once then
> took a snapshot of the memory (after several forced gc's using the UI)
> then i ran the client again and took a second snapshot (again after
> several forced gc's using the UI) and i did not see any leaks.... :(
> 
> thanks,
> dims
> 
> 
> 
> On 17 Jul 2004 15:10:56 +0530, Nishant Kumar <nishant.kumar@itellix.com> wrote:
> > hi Peter,
> >         i did some more tests in this regard and have attached some interesting
> > gc log images. let me first remove some small misunderstandings in my
> > last mail.
> >         i used POST just to reduce the cost at client. and i was running
> > simultaneous 8 clients. and all i wanted to say was that the server (1.2
> > beta) didn't end up with OOM.
> >         secondly i insisted on using two separate machines just because i don't
> > have the luxury of 2GB machines.:(
> >         now coming to you third point about the message size. i totally agree
> > with your point that the message size should be large to see this error
> > as soon as possible and i also think you have done a great job writing
> > out such a good test case. but it somehow doesnot result in OOM.
> >
> > so finally what have i done.
> >
> >         firstly i figured out a big memory hogger. the ByteArray class.
> > axis team could have chosen to use
> > http://jakarta.apache.org/commons/io/apidocs/org/apache/commons/io/output/ByteArrayOutputStream.html
> > or my own (having same functionality)
> > http://nishantkumar.com/mycode/codedocs/nkio/com/nk/io/ByteArrayOutputStream.html
> > instead. javadocs explains the difference. so i replaced the byte array
> > class with the second class mentioned above.
> >
> >         to see the memory usage pattern i ran your client in a single thread. i
> > captured the verbose gc output and viewed the same using gcviewer.
> > memory allocated was 128M.
> >         the first observation is that the unaltered code causes many full GC to
> > run. you can see the same in gc2.png.
> >         so i changed the class i mentioned above and ran the test again and
> > this time there were far less full gc. see gc8.png.
> >         but as you can see in this graph, the graph keeps on increasing till a
> > full gc. i was expecting the graph to come down as soon one request was
> > handled and the next request began (i ran the client in one thread
> > remember). the reason for this was that since the byte array within the
> > ByteArray (even the modified one) class has lasted for so many small gc,
> > it has been promoted to the old generation space and was waiting for a
> > full gc to reclaim it.
> >         to counter this i ran the server with -Xconcgc, concurrent gc. see the
> > result in gc6.png. in this case the graph dipped after each request.
> > note that i had run this test for a smaller time.
> >         since i had to observe these things, i had to run this test with single
> > client. so what i can conclude from this limited test is that just
> > because memory seems to be rising, it doesn't mean that there is memory
> > leak, only when a Full GC is unable to recover the same can we say that
> > there is some problem. System.gc() is not guaranteed to run gc, not even
> > a partial gc.
> >         axis team should think about changing the ByteArray class, if not for
> > memory reason then because there are many bugs in the class. using file
> > based backup in this class is again not a good idea or not nicely
> > implemented.
> >         hope this helps.
> > thanks,
> > nishant
> >
> > On Fri, 2004-07-16 at 23:52, Peter Molettiere wrote:
> > > It's fine to try this test using a simple unix POST command, however,
> > > you must make your client send multiple simultaneous requests. We've
> > > done a lot of testing to isolate this server side memory leak, and
> > > found that it only occurs when multiple client requests are being
> > > handled at the same time on the server. You should be able to run all
> > > threads indefinitely without running into a server side OOM error. Just
> > > being able to run all eight threads once doesn't prove the leak isn't
> > > there.
> > >
> > > As far as your assertion that you must run the client on a different
> > > machine than the server, I don't see why this is a requirement. I can
> > > run the client with 1G of heap, and the server with 1G of heap, and
> > > still have plenty of physical ram left over for the OS and other apps.
> > > (Yes, I have an insane amount or memory in my machine.) The client and
> > > the server both run in completely separate VMs, so there should be no
> > > issue running the tests this way. If you still think there is a valid
> > > reason for needing to run the test on separate machines, I'm all ears.
> > >
> > > As far as your last point about the size of the messages we're dealing
> > > with. This test case is meant to display a server side memory leak, not
> > > that serialization/deserialization in axis uses extreme amounts of
> > > temporary memory (which it does, but we can live with that). The bug
> > > (memory leak on the server) can be replicated with much smaller return
> > > response messages, but it takes longer to notice. We notice pretty
> > > quickly with our application, since we do send dynamic messages of this
> > > size occasionally.
> > >
> > > If you run in a profiler, like JProfiler, you'll see the server memory
> > > use growing very plainly.
> > >
> > > --Peter
> > >
> > > On Jul 15, 2004, at 10:09 PM, Nishant Kumar wrote:
> > >
> > > > hi Peter,
> > > >     i tried looking into this matter. let me first explain what i modified
> > > > in your test case.
> > > >
> > > > firstly i replaced your java client by a simple UNIX POST command.
> > > >
> > > > something like this.
> > > >
> > > > POST -H 'Content-Type: text/xml; charset=utf-8' -H 'Accept:
> > > > application/soap+xml, application/dime, multipart/related, text/*' -H
> > > > 'User-Agent: Axis/1.2beta2' -H 'Host: localhost:8080' -H
> > > > 'Cache-Control:
> > > > no-cache' -H 'Pragma: no-cache' -H 'SOAPAction: ""'
> > > > http://10.0.2.124:8080/axis/services/MemoryTester <getRoot.xml >
> > > > response
> > > >
> > > > where getRoot.xml has the packet you used to send by your java client.
> > > >
> > > > the benefit of doing this was that one can avoid having deserialization
> > > > cost at the client.
> > > >
> > > > i hope you were anyway running the client on a different machine than
> > > > the server. this is a MUST.
> > > >
> > > > you can run with verbosegc to find out how much memory the server is
> > > > actually taking. no need to run gc.
> > > >
> > > > after doing this i was able to run 8 such concurrent client with just
> > > > 300M set as -mx.
> > > >
> > > > another thing i noted was that the response was 13MB in size. that is
a
> > > > huge response, that too when you are generating the same and not
> > > > serving
> > > > some static file.
> > > >
> > > > thanks,
> > > > nishant
> > > >
> > > >
> > > > On Wed, 2004-07-14 at 23:53, Peter Molettiere wrote:
> > > >> I reported 1423, with the test case -- If there's anything I can do
to
> > > >> help you out at all, please let me know. I'm more than happy to help.
> > > >>
> > > >> --Peter
> > > >>
> > > >> On Jul 14, 2004, at 6:21 AM, Davanum Srinivas wrote:
> > > >>
> > > >>> Glen,
> > > >>>
> > > >>> I'd REALLY like your help with the following:
> > > >>> http://nagoya.apache.org/jira/browse/AXIS-1120
> > > >>> (DefaultTypeMappingImpl
> > > >>> is not compliant with JAX-RPC 1.1)
> > > >>> http://nagoya.apache.org/jira/browse/AXIS-1423 (OutOfMemoryError
in
> > > >>> axis with multi-threaded usage)
> > > >>> http://nagoya.apache.org/jira/browse/AXIS-1391 (severe memory
leakage
> > > >>> in server side service (with tomcat))
> > > >>>
> > > >>> 1120 involves exposing the currently internal flag
> > > >>> (jaxrpc11Compliance) via the various command line utils and ant
> > > >>> tasks.
> > > >>> the other two 1423 and 1391 are severe problems with performance
with
> > > >>> test cases that recreate the problem.
> > > >>>
> > > >>> Thanks,
> > > >>> dims
> > > >>>
> > > >>> On Wed, 14 Jul 2004 08:44:33 -0400, Glen Daniels
> > > >>> <glen@thoughtcraft.com> wrote:
> > > >>>> dims:
> > > >>>>
> > > >>>> Can you please let us know which bugs those were?  I'd like
to take
> > > >>>> another
> > > >>>> brief look at them during beta-2.
> > > >>>>
> > > >>>> --Glen
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> ----- Original Message -----
> > > >>>> From: "Davanum Srinivas" <davanum@gmail.com>
> > > >>>> To: <axis-dev@ws.apache.org>
> > > >>>> Sent: Wednesday, July 14, 2004 8:32 AM
> > > >>>> Subject: Re: When is Axis 1.2 expected to be "final"?
> > > >>>>
> > > >>>> WSE 2.0 is FINAL now!!!! (see
> > > >>>> http://msdn.microsoft.com/webservices/building/wse/). Also
changed
> > > >>>> the
> > > >>>> priority for the blockers to major.
> > > >>>>
> > > >>>> -- dims
> > > >>>>
> > > >>>> On Wed, 14 Jul 2004 13:32:24 +0200, Banck, Arent-Jan
> > > >>>> <ajbanck@informatica.com> wrote:
> > > >>>>> As .NET 2.0 is only a beta, 1.1 is the most recent .NET
version.
> > > >>>>>
> > > >>>>> There are currently 5 cases marked as blocker in Jira.
If these are
> > > >>>>> not
> > > >>>> blockers for 1.2 they should probably be given another priority.
> > > >>>>>
> > > >>>>> - Arent-Jan
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Davanum Srinivas [mailto:davanum@gmail.com]
> > > >>>>> Sent: Wednesday, July 14, 2004 1:22 PM
> > > >>>>> To: axis-dev@ws.apache.org
> > > >>>>> Subject: Re: When is Axis 1.2 expected to be "final"?
> > > >>>>>
> > > >>>>> Thomas,
> > > >>>>>
> > > >>>>> IF someone submits a test case that displays this problem
with
> > > >>>>> latest .NET
> > > >>>> 2.0 FINAL and Axis LATEST CVS, then am willing to hold up
the beta.
> > > >>>> Not
> > > >>>> otherwise.
> > > >>>>>
> > > >>>>> -- dims
> > > >>>>>
> > > >>>>> On Wed, 14 Jul 2004 10:07:48 +0200, Thomas Börkel <tbo@ap-ag.com>
> > > >>>>> wrote:
> > > >>>>>> HI!
> > > >>>>>>
> > > >>>>>> Sorry, but how can Axis 1.2 going to be released,
if there are
> > > >>>>>> severe
> > > >>>>>> interop problems with .NET?
> > > >>>>>>
> > > >>>>>> http://nagoya.apache.org/jira/browse/AXIS-1308
> > > >>>>>>
> > > >>>>>> Thomas
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: Tom Jordahl [mailto:tomj@macromedia.com]
> > > >>>>>>> Sent: Freitag, 9. Juli 2004 17:03
> > > >>>>>>> To: 'axis-dev@ws.apache.org'
> > > >>>>>>> Subject: RE: When is Axis 1.2 expected to be "final"?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Real soon now....
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Tom Jordahl
> > > >>>>>>> Macromedia Server Development
> > > >>>>>>>
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: ricky_frost@peoplesoft.com
> > > >>>>>>> [mailto:ricky_frost@peoplesoft.com]
> > > >>>>>>> Sent: Thursday, July 08, 2004 1:18 PM
> > > >>>>>>> To: axis-dev@ws.apache.org
> > > >>>>>>> Subject: When is Axis 1.2 expected to be "final"?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> A project I'm looking at (ws-wsrp4j) just incorporated
the 1.2
> > > >>>>>>> beta
> > > >>>>>>> and so I would like to know. I asked on that list
but they
> > > >>>>>>> referred
> > > >>>>>>> me here...
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> Davanum Srinivas - http://webservices.apache.org/~dims/
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> Davanum Srinivas - http://webservices.apache.org/~dims/
> > > >>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Davanum Srinivas - http://webservices.apache.org/~dims/
> > > >>
> > >
> >
> >
> >
> 
> --
> Davanum Srinivas - http://webservices.apache.org/~dims/
> 


-- 
Davanum Srinivas - http://webservices.apache.org/~dims/

Mime
View raw message