axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Molettiere (JIRA)" <axis-...@ws.apache.org>
Subject [jira] Commented: (AXIS-1771) Excessive Memory Use During Serialization/Deserialization
Date Wed, 26 Jan 2005 18:05:20 GMT
     [ http://issues.apache.org/jira/browse/AXIS-1771?page=comments#action_58098 ]
     
Peter Molettiere commented on AXIS-1771:
----------------------------------------

Hmm... I've spent the morning looking at your revised test, Venkat, but it doesn't seem to
hold up.

First, let me say that yes, your test as written, for the one small message size that you
test, does produce the number you're claiming. But for larger message sizes, I'm seeing deserialization
memory usage ratios as high as 42 to 1. 

However, when I pull your reporting and garbage collection changes back into my test, and
run over a range of message sizes, your ratio doesn't remain constant, even for the same message
size that you test.

In answer to your question about what ratio is good enough, I'll say this. If you could in
fact get the ratio down to 15 to 1 it would be an improvement over the current situation,
and would be much appreciated, but O(15n) performance is not ideal.

Here's what I'm seeing, for a few cases. You can run my code (will attach in a minute) to
see all cases.

     [java] Created tree with 5 levels and 3 children at each level
     [java] Total memory usage: Created tree = 912 KBytes (934232)
     [java] Increase in allocation since last check = 229 KBytes (234736)

     [java] Total memory usage: Serialized tree = 4 MBytes (4958272)
     [java] Increase in allocation since last check = 3 MBytes (4024040)

     [java] Total memory usage: Deserialized tree. = 13 MBytes (13692168)
     [java] Increase in allocation since last check = 8 MBytes (8733896)

The above test represents a ratio of 13 to serialize, and 35 to deserialize.

     [java] Created tree with 5 levels and 4 children at each level
     [java] Total memory usage: Created tree = 1 MBytes (1908112)
     [java] Increase in allocation since last check = 920 KBytes (942464)

     [java] Total memory usage: Serialized tree = 27 MBytes (28685800)
     [java] Increase in allocation since last check = 25 MBytes (26777688)

     [java] Total memory usage: Deserialized tree. = 66 MBytes (69915592)
     [java] Increase in allocation since last check = 39 MBytes (41229792)

The above test represents a ratio of 27 to serialize, and 42 to deserialize.

     [java] Created tree with 5 levels and 5 children at each level
     [java] Total memory usage: Created tree = 5 MBytes (6065264)
     [java] Increase in allocation since last check = 4 MBytes (4247184)

     [java] Total memory usage: Serialized tree = 95 MBytes (100226664)
     [java] Increase in allocation since last check = 89 MBytes (94161400)

     [java] Total memory usage: Deserialized tree. = 246 MBytes (258236984)
     [java] Increase in allocation since last check = 150 MBytes (158010320)

The above test represents a ratio of 22 to serialize, and 38 to deserialize.



> Excessive Memory Use During Serialization/Deserialization
> ---------------------------------------------------------
>
>          Key: AXIS-1771
>          URL: http://issues.apache.org/jira/browse/AXIS-1771
>      Project: Axis
>         Type: Bug
>   Components: Basic Architecture
>     Versions: 1.2RC2
>  Environment: JDK 1.4.2, Mac OS X, Linux, Windows
>     Reporter: Peter Molettiere
>     Assignee: Venkat Reddy
>  Attachments: MTC_venkat.java, MemoryTesterClient.java, SOAPmsg_multiref_false.xml, SOAPmsg_multiref_true.xml,
SerializationContext.java.diff, memory-use-test.tgz, recursion.png
>
> Axis uses pathological amounts of memory during the serialization/deserialization process.
> We see about a 30 to 1 ratio of memory used during (de)serialization to in-memory representation
of the objects being (de)serialized. So ser/deser in axis of a 2M graph of objects uses 288M
of memory! Further, the memory used seems to scale linearly with the size of the object graph
being serialized.
> The memory used does seem to be released once serialization is done, so this isn't a
leak.
> Using the attached example code, (based on the code used to demonstrate AXIS-1423) you
can see this behavior. The test automatically runs with a max heap size of 1024M, and runs
out of memory serializing a 28M object graph. 
> As provided, it generates the following output:
> Buildfile: build.xml
> build:
>     [javac] Compiling 1 source file to /Users/pietro/Work/Axis Memory Test/build/classes
> run:
>      [java] - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart).
Attachment support is disabled. 
>      [java] Created tree with 5 levels and 3 children at each level
>      [java] Axis used 13 MBytes to serialize 230 KBytes, a ratio of 30.0
>      [java] GC freed 13 MBytes
>      [java] Created tree with 5 levels and 4 children at each level
>      [java] Axis used 71 MBytes to serialize 1 MBytes, a ratio of 31.0
>      [java] GC freed 71 MBytes
>      [java] Created tree with 5 levels and 5 children at each level
>      [java] Axis used 288 MBytes to serialize 2 MBytes, a ratio of 51.0
>      [java] GC freed 287 MBytes
>      [java] Created tree with 5 levels and 6 children at each level
>      [java] Axis used 671 MBytes to serialize 11 MBytes, a ratio of 29.0
>      [java] GC freed 675 MBytes
>      [java] Created tree with 5 levels and 7 children at each level
>      [java] Out of Memory serializing 28 MBytes tree.
>      [java] Java Result: 1
> BUILD SUCCESSFUL
> Total time: 2 minutes 51 seconds
> Note that the ratios are halved from the reported values, since it includes both serialization
and deserialization of the object graph. So axis uses 30.5M to serialize a 1M message, and
another 30.5M to deserialize it, resulting in the reported 71M reported above. Also, notice
that the ratio stays close to 30 to 1 regardless of object graph size. This is the linear
scaling I mention above.
> Note also, that if you tweak the code to generate very small object graphs, you see extremely
high ratios, but I would expect this due to simple one-time overhead to operate on very small
amounts of data. That's why I start with the graph size that I do.
>  

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
   http://www.atlassian.com/software/jira


Mime
View raw message