ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vkulichenko <valentin.kuliche...@gmail.com>
Subject Re: Memory leak or mis-configured grid/Ignite cluster?
Date Mon, 22 Jun 2015 23:03:03 GMT
Darren,

This is not a memory leak. Ignite's OptimizedMarshaller internally maintains
a set of thread local byte arrays which are used to serialize objects. This
improves performance because we don't have to allocate these arrays each
time, but consumes memory.

You ran out of memory because task result is very large. Most likely each
execution happened in different thread, and each of this threads allocated
its own byte array (and each of them was at least 430mb).

This can be resolved by configuring pooling of streams in marshaller:

<property name="marshaller">
    <bean
class="org.apache.ignite.marshaller.optimized.OptimizedMarshaller">
        <property name="poolSize" value="2"/>
    </bean>
</property>

This will limit the number of created buffers to provided value. But note
that if you execute tasks in parallel, this can cause performance
degradation because different threads will use the same buffer to serialize
the response.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Memory-leak-or-mis-configured-grid-Ignite-cluster-tp556p562.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message