ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Reusing Ignite-Marshaller leads to a Memory-Leak
Date Sun, 22 Jan 2017 05:15:21 GMT
Sorry, didn’t pay attention that the size of every object is 1 KB. Then I would suggest to
share your test/heap dump to see what happens.


> On Jan 21, 2017, at 9:10 PM, Denis Magda <dmagda@apache.org> wrote:
> Hello Peter,
> This behavior is expected. Look, you serialize 2000 objects * 1000 KB which gives ~ 2
GB in total. After that you hold the references to these objects by putting them into an ordinary
hash map. The objects will not be removed from the heap until the map keeps them.
> One the other side, when the same objects are placed into an off-heap cache they will
be collected from the heap because at the end there will be no reachable references to them.
> —
> Denis
>> On Jan 21, 2017, at 2:46 PM, Peter Schmitt <peter.schmitt.jee@gmail.com <mailto:peter.schmitt.jee@gmail.com>>
>> Hello Ignite-Community!
>> I would like to use the Marshaller from Ignite for de-/serialising objects.
>> I would like to re-use it, because it's very fast and I'm using Ignite anyway.
>> I tried it with:
>> //every thread gets the Marshaller via:
>> GridKernalContext ctx = ((IgniteKernal) ignite).context();
>> Marshaller marshaller = ctx.grid().configuration().getMarshaller();
>> ->
>> byte serializedResult[] = marshaller.marshal(obj);
>> or
>> Object deserializedResult = marshaller.unmarshal(serializedResult, ctx.getClass().getClassLoader());
>> In a simple test it worked very well (and fast).
>> However, once I'm using larger Objects (e.g. 2000 instances each ~1kb after marshalling)
and store the results in a map, the Heap consumption grows a lot (almost 2gb in few seconds).
>> If I store the same objects in an Ignite OffHeap-Cache, the (OffHeap) memory-consumption
is minimal.
>> In the debugger I saw that the heap grows once those 2000 instances get processed.
>> (A manually triggered GC can just release few mb.)
>> I even tried to re-use the marshaller, but nothing changed.
>> At runtime BinaryMarshaller is used which wraps GridBinaryMarshaller.
>> Any hint to get rid of the leak is appreciated,
>> Peter

View raw message