ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Scherbakov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-10920) Optimize HistoryAffinityAssignment heap usage.
Date Wed, 23 Jan 2019 19:37:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-10920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750373#comment-16750373
] 

Alexei Scherbakov commented on IGNITE-10920:
--------------------------------------------

[~kbolyandra],

My comments:
 # Please use already existing property {{org.apache.ignite.IgniteSystemProperties#IGNITE_DISABLE_AFFINITY_MEMORY_OPTIMIZATION}} to
completely disable new optimization in case if something goes wrong.
 # It's necessary to undestand if the optimization actually helps. Please add new benchmark
using jol library and compare heap usage for HistoryAffinityAssignment with enabled optimization
and not. Look at {{org.apache.ignite.internal.benchmarks.jol.GridAffinityAssignmentJolBenchmark}} as
reference. Possible test scenario could be adding new node to topology until it grows up to some
threshold, with different number of partitions, and measure instances size. Memory utilization
should always be better or the same with enabled optimization.
 # Looks like creating an assignment copy could be avoided in {{org.apache.ignite.internal.processors.affinity.HistoryAffinityAssignment#idealAssignment}}}}.
Just create a List<List<ClusterNode>> array view instead.

> Optimize HistoryAffinityAssignment heap usage.
> ----------------------------------------------
>
>                 Key: IGNITE-10920
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10920
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Alexei Scherbakov
>            Assignee: Konstantin Bolyandra
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> With large topology and large amount of caches/partitions many server discovery events
may quickly produce large affinity history, eating gigabytes of heap.
> Solution: implement some kind of a compression for affinity cache map.
> On example, affinity history could be stored as delta to some previous version.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message