axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nishant Kumar <nishant.ku...@itellix.com>
Subject Re: [jira] Commented: (AXIS-1771) Excessive Memory Use During Serialization/Deserialization
Date Thu, 20 Jan 2005 05:14:53 GMT
Peter,

yes I was using testTree(5,3).

serialization cannot be improved much as the generated xml was itself
1.1 Mb. which is roughly the same amount of memory being used by the
server during serialization.

what can be improved in serialization is the way it is serialized so
that the deserialization process is more efficient. I have already added
a patch for one such thing in your bug report.

some more bad news. the amount you have measured is just the amount of
memory that axis server holds on till the end while de-serializing.
during thing process it creates may local objects that is GCed while
deserialization process. If the -Xloggc of sun jvm is to be believed
then the amount of memory wasted this way is 23MB for testTree(5,3).
This 23MB only includes the memory churned by short lived objects. This
excludes the 11MB that you finally measure.

In this respect what venkat suggested will be good to look at. If he can
explain the usage of SAX2EventRecorder may be I can also have a look.

>>From a glance it seemed to me that message is first parsed as DOM,
creating MessageElements (RPCElement in your case) and then they are
converted to objects. If this is what happens (which I suspect) then how
is SAX involved.

thanks,
nishant


On Thu, 2005-01-20 at 00:59, Peter Molettiere wrote:
> Yes, I've double checked, and you're right, deserialization does take 
> less memory than serialization. I'd suggest though that the numbers for 
> serialization could be improved as well, though. I'm not sure what tree 
> size you used to generate your 1M ser to 12M deser numbers... I'm 
> guessing it was smaller than 5,3. And it doesn't seem to scale any 
> differently than deserialization.
> 
>  From my new test, replicating your separation of ser and deser:
> > Created tree with 5 levels and 3 children at each level
> > Axis used 4 MBytes to serialize 230 KBytes, a ratio of 20.0
> > Axis used 14 MBytes to deserialize 230 KBytes, a ratio of 64.0
> > Created tree with 5 levels and 4 children at each level
> > Axis used 33 MBytes to serialize 1 MBytes, a ratio of 29.0
> > Axis used 70 MBytes to deserialize 1 MBytes, a ratio of 63.0
> > Created tree with 5 levels and 5 children at each level
> > Axis used 138 MBytes to serialize 2 MBytes, a ratio of 47.0
> > Axis used 269 MBytes to deserialize 2 MBytes, a ratio of 92.0
> > Created tree with 5 levels and 6 children at each level
> > Axis used 277 MBytes to serialize 7 MBytes, a ratio of 36.0
> > Axis used 675 MBytes to deserialize 7 MBytes, a ratio of 89.0
> > Created tree with 5 levels and 7 children at each level
> > Axis used 570 MBytes to serialize 28 MBytes, a ratio of 20.0
> > Out of Memory deserializing 28 MBytes tree.
> 
> Imagine a server trying to handle multiple simultaneous requests to 
> serialize a 28M tree -- say, 8. That would require over 4.5G of heap, 
> something not supportable on any 32 bit JVM. And 8 simultaneous 
> requests is not a very high scalability target. Granted there are 
> things to do to work around this, like trying to break your request 
> into smaller units, etc., but that doesn't change the scaling factor 
> for using axis.
> 
> If there's anything I can do to help, please let me know!
> 
> --
> Peter Molettiere
> Senior Engineer
> Truereq, Inc.
> http://www.truereq.com/
> 


Mime
View raw message