axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Molettiere <pie...@axonstudios.net>
Subject Re: Axis - Out OF Memory problem
Date Tue, 18 Jan 2005 04:19:49 GMT

We find that it's a straight 10 to 1 ratio between the in-memory 
representation and the memory used to serialize it, and this holds true 
for pretty much any size message. It's not a problem when you're 
dealing with 1 or 2 M of in-memory representation, since that runs to 
only 10 or 20M during serialization. I don't remember what the default 
heap size is off the top of my head, but it only takes a 7M in memory 
representation to exceed a 64M heap. Obviously 32 is even easier to 
break. This is the source of the common advice "raise your heap size" 
in response to all the messages posted to this list claiming out of 
memory exceptions during serialization.

If you look back at my original jira bug posting, there's a test case 
which will let you configure a tree of arbitrary width and depth. Make 
a big one, and watch memory use during serialization in a profiler. 
Checkpoint it before and watch for the high water mark, and you'll see 
what I'm talking about.

That bug was about memory leaks, and those seem to be mostly fixed. 
Anything left seems small enough that we can get 2 month up times now, 
which we're happy with. If you GC after you serialize you see gobs of 
memory being released.

--Peter

On Jan 17, 2005, at 11:25 AM, pawan.v.udernani@gsk.com wrote:

>
> Hi,
>
> Does anyone has an idea  how "large" are we talking about ? Do we have 
> any data that says that it fails for certain size of objects, in terms 
> of bytes or Kbytes ? If yes, I can compare it to the size of data that 
> our webservice client consumes as of now.
>
>
> Thanks,
>
> Pawan.
>
>
>
>
>
>
> "Davanum Srinivas" <davanum@gmail.com>
>
> 17-Jan-2005 14:21
>
> Please respond to axis-dev@ws.apache.org; Please respond to 
> dims@apache.org
>
>
>        
>
>  To
> axis-dev@ws.apache.org
>
> cc
>
> Subject
> Re: Axis - Out OF Memory problem
>
>
>
>
>
>
>
> Peter,
>
>  If possible, We'd like a test case with large object graphs to
>  reproduce the problem...So that we can make sure we can recreate the
>  problem and fix it as well.
>
>  thanks,
>  dims
>
>
>  On Mon, 17 Jan 2005 10:44:09 -0800, Peter Molettiere
>  <pietro@axonstudios.net> wrote:
>  >
>  > Not sure what you'd like me to do. The method we're using to deal 
> with
>  > serializing large graphs of objects through axis is clearly 
> outlined in
>  > my previous posts. If there's something in particular you'd like us 
> to
>  > look at, let me know.
>  >
>  > Last night as I was falling asleep I realized I didn't mention 
> another
>  > possibility, though, so I may as well mention that here, too.
>  > Basically, if you're having trouble serializing large graphs of
>  > objects, then you can try re-writing your application to issue many
>  > small requests where you used to issue a single large request. The 
> most
>  > common recommendation seen is in response to people trying to 
> serialize
>  > large result sets. In that case it's pretty easy to send X rows, and
>  > request the next X rows in another request.
>  >
>  > Our application hasn't been so easily tamed, since we're serializing
>  > java object graphs, usually trees. We've talked about implementing a
>  > serialization scheme which would allow us to request any subtree of
>  > depth N from any given node, which would allow us to fetch our tree
>  > with a series of separate calls, but we've had a lot of other more
>  > important tasks, and the heap size and gc tweaks I've already 
> mentioned
>  > are working well enough for us at the moment.
>  >
>  > The other method we've been considering, since interop isn't much 
> of a
>  > requirement for us, is to implement a simple XML-RPC mechanism using
>  > standard XML serialization and plain vanilla HTTP servlets.
>  >
>  > --Peter
>  >
>  > On Jan 15, 2005, at 7:24 AM, Davanum Srinivas wrote:
>  >
>  > > Peter,
>  > >
>  > > anyway you can help us tune latest axis cvs?
>  > >
>  > > thanks,
>  > > dims
>  > >
>  > >
>  > > On Fri, 14 Jan 2005 14:04:31 -0800, Peter Molettiere
>  > > <pietro@axonstudios.net> wrote:
>  > >>
>  > >> On Jan 14, 2005, at 1:29 PM, pawan.v.udernani@gsk.com wrote:
>  > >>> Your mail provides a great deal of information. But, now I feel
>  > >>> trapped. Thanks anyways.
>  > >>
>  > >> /shrug/
>  > >>
>  > >> Try these, in this order:
>  > >>
>  > >> 0) Use the latest axis distro
>  > >> 1) Use xerces 2.6.2
>  > >> 2) Up your heap size:  -Xmx768M -Xms128M -XX:+UseConcMarkSweepGC
>  > >>
>  > >> if the above don't work, then
>  > >>
>  > >> 3) Write your own SOAP engine.
>  > >>
>  > >> That's about the shape of it, I think.
>  > >>
>  > >> --
>  > >> Peter Molettiere
>  > >> Senior Engineer
>  > >> Truereq, Inc.
>  > >> http://www.truereq.com/
>  > >>
>  > >>
>  > >
>  > >
>  > > --
>  > > Davanum Srinivas - http://webservices.apache.org/~dims/
>  > >
>  > --
>  > Peter Molettiere
>  > Senior Engineer
>  > Truereq, Inc.
>  > http://www.truereq.com/
>  >
>  >
>
>
>  --
>  Davanum Srinivas - http://webservices.apache.org/~dims/
>
>
>
--
Peter Molettiere
Senior Engineer
Truereq, Inc.
http://www.truereq.com/


Mime
View raw message