ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: Unexpected heap usage by GridDhtPartitionTopologyImpl
Date Thu, 07 Apr 2016 22:54:38 GMT
Tolga,

The cleanup threads "ttl-cleanup-workers" are used to eagerly remove 
expired cache entries. Expiration policy can be set either a cache wide 
in CacheConfiguration or can be used later with 
cache.withExpirePolicy(...) calls.
I failed to reproduce your case. What I've done is started 30 caches and 
destroyed all of them later. Visual VM showed that all 
"ttl-cleanup-workers" were stopped successfully.
What Ignite version do you use?

In any case if you are not planing to use expiration policy you can set 
CacheConfiguraiton.setEagerTtl to false and the ttl workers Threads 
won't be created at all.

Regards,
Denis

On 4/7/2016 3:43 PM, Tolga Kavukcu wrote:
> Hi Denis,
>
> IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE parameter seems like decreased 
> heap usage. I will run longer tests to check heap behaviour.
>
> Also i need another help with thread's created by ignite. I found out 
> that ignite creates a cleanup thread named "ttl-cleanup-worker" for 
> each cache.  But when cache is destroyed, clean up thread does not 
> deleted. Instead it waits sleeping state at all.
>
> My first question is that , is it possible to decrease thread count 
> with a configuration, like "thread pool with x threads" for all caches.
> Secondly, is "unremoved threads" are expected behaviour.
>
> Thanks.
>
> On Thu, Apr 7, 2016 at 2:40 PM, Denis Magda <dmagda@gridgain.com 
> <mailto:dmagda@gridgain.com>> wrote:
>
>     Hi Tolga,
>
>     GridDhtPartitionTopologyImpl is created per cache. If you destroy
>     a cache this object should be GCed. However you should use
>     cache.destroy() for that.
>
>     Please also make sure that you make "live set" heap dumps only.
>     Try to perform GC explicitly before making the dump because a
>     collector may clean dead objects much later depending on its
>     heuristics.
>
>     --
>     Denis
>
>     On 4/7/2016 8:27 AM, Tolga Kavukcu wrote:
>>     Hi Denis,
>>
>>     Thanks for the response. I will try
>>     IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE parameter. The screnshots
>>     taken from eclipse memory analyser which opens and analyses heap
>>     dump. I understand heap requirement for wrapping and indexing
>>     off-heap entry positions. But also found out that instances of
>>     /org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl
>>     /is constantly increasing within jvm.
>>
>>
>>     I also create and destroy so many small caches during the
>>     lifecycle, do you think that it is possible to destroyed caches
>>     leaves a footprint in heap.
>>
>>     The previous scrreenshots was dominator tree view of memory
>>     analyser. I attached again with headers.
>>
>>      You can see that each of GridDhtPartitionTopologyImpl uses 20mb~
>>     heap. And there are 72 instances of GridDhtPartitionTopologyImpl
>>     living.
>>
>>     Also i attached screenshots of leak suspect report of memory
>>     analyzer, which is taken periodically. You an see that instances
>>     of /GridDhtPartitionTopologyImpl keeps increasing. /
>>
>>     Any ideas or suggestions?
>>
>>     On Wed, Apr 6, 2016 at 6:00 PM, Denis Magda <dmagda@gridgain.com
>>     <mailto:dmagda@gridgain.com>> wrote:
>>
>>         Hi Tolga,
>>
>>         GridDhtPartitionTopologyImpl contains list of partitions that
>>         belong to a specific node. In case of offheap caches each
>>         partition (concurrent map) contains set of wrappers around
>>         keys->values, stored offheap. The wrapper holds information
>>         that's needed to unswap a value or a key to Java heap from
>>         offheap when required by a user application.
>>         So Ignite requires extra space for internal needs even when
>>         offheap mode is used.
>>
>>         I would recommend you trying to reduce
>>         IgniteSystemProperties.IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE.
>>         This is the size of the queue that keeps deleted entries for
>>         internal needs as well.
>>         https://apacheignite.readme.io/v1.5/docs/capacity-planning
>>
>>         BTW, could you explain what columns from your screenshot mean
>>         exactly? What tool did you use to create the memory snapshot?
>>
>>         --
>>         Denis
>>
>>
>>
>>         On 4/6/2016 3:02 PM, Tolga Kavukcu wrote:
>>>         Hi everyone,
>>>
>>>         I use partitioned ignite cache for a very dynamic data.
>>>         Means that there are many updates,deletes and puts with
>>>         around 5m~ row.
>>>
>>>         So to avoid gc pauses i use off-heap mode. But when i
>>>         analyse heap i see that count and heap size of
>>>         /org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl/
is
>>>         increasing constantly.
>>>
>>>         Please see attached screenshots taken from mat heap dump.
>>>
>>>         <bean
>>>         class="org.apache.ignite.configuration.CacheConfiguration"
>>>         name="DEFAULT"> <property name="atomicityMode"
>>>         value="ATOMIC" /> <property name="cacheMode"
>>>         value="PARTITIONED" /> <property name="memoryMode"
>>>         value="OFFHEAP_TIERED" /> <property name="backups" value="1"
>>>         /> <property name="affinity"> <bean
>>>         class="org.apache.ignite.cache.affinity.fair.FairAffinityFunction">
>>>         <constructor-arg index="0" type="int" value="6"/> </bean>
>>>         </property><property name="writeThrough" value="false" />
>>>         <property name="writeBehindEnabled" value="false" /> </bean>
>>>         Thanks for helping out.
>>>         There are totally 1.2 heap used by
>>>         GridDhtPartitionTopologyImpl, almost equals to my data size.
>>>         Do you think that there are problems with configuration.
>>>
>>>         *Tolga KAVUKÇU
>>>         *
>>
>>
>>
>>
>>     -- 
>>     *Tolga KAVUKÇU
>>     *
>
>
>
>
> -- 
> *Tolga KAVUKÇU
> *


Mime
View raw message