axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Molettiere <pie...@axonstudios.net>
Subject Re: [jira] Commented: (AXIS-1771) Excessive Memory Use During Serialization/Deserialization
Date Thu, 20 Jan 2005 17:46:12 GMT

First, thanks for looking into this. I appreciate your efforts to 
improve the situation.

On Jan 19, 2005, at 9:14 PM, Nishant Kumar wrote:
> yes I was using testTree(5,3).
>
> serialization cannot be improved much as the generated xml was itself
> 1.1 Mb. which is roughly the same amount of memory being used by the
> server during serialization.

I don't see the same amount of memory you do. I'm seeing 4M used during 
serialization. I'd like to understand why we're seeing different 
numbers for the same test. For the 5,3 test, here's what I'm seeing for 
serialization:

>>  From my new test, replicating your separation of ser and deser:
>>> Created tree with 5 levels and 3 children at each level
>>> Axis used 4 MBytes to serialize 230 KBytes, a ratio of 20.0


> some more bad news. the amount you have measured is just the amount of
> memory that axis server holds on till the end while de-serializing.
> during thing process it creates may local objects that is GCed while
> deserialization process.

This is exactly the problem I'm reporting. The memory axis uses during 
the deserialization process is an extraordinary multiple of the size of 
the object graph being deserialized. If my numbers are to be believed, 
it's more than 60 to 1. While this memory is released at the end of the 
deserialization process, this multiple creates what I believe to be an 
artificially low limit on the size of message axis can deal with for 
any given max heap size. For a 4 GIG heap (the largest you could 
possibly get under any 32 bit OS) the largest message axis could 
potentially deserialize is only 71 MEGs. In a default JVM running with 
-server, the heap is usually only 128M, which means  you can't 
deserialize anything larger than about 2M.  Adding an additional 384M 
of heap only nets you an extra 6M in max message size.

Doesn't the amount of memory used to deserialize things seem a little 
out of hand to you?

> If the -Xloggc of sun jvm is to be believed
> then the amount of memory wasted this way is 23MB for testTree(5,3).
> This 23MB only includes the memory churned by short lived objects. This
> excludes the 11MB that you finally measure.

Whatever it is that I'm measuring, it's preventing the use of axis for 
the serialization of large messages.

> In this respect what venkat suggested will be good to look at. If he 
> can
> explain the usage of SAX2EventRecorder may be I can also have a look.
>
>> From a glance it seemed to me that message is first parsed as DOM,
> creating MessageElements (RPCElement in your case) and then they are
> converted to objects. If this is what happens (which I suspect) then 
> how
> is SAX involved.

Certainly any improvement will be greatly appreciated.

--
Peter Molettiere
Senior Engineer
Truereq, Inc.
http://www.truereq.com/


Mime
View raw message