ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taras Ledkov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
Date Wed, 04 May 2016 15:15:12 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270799#comment-15270799

Taras Ledkov commented on IGNITE-3018:

Please take a look at the attached heatmaps. The node distributions of affinity function with
MD5 hash & Wang hash with bucket based algorithm are compared for 3, 64, 100, 128, 200,
..., 600 nodes.

Horizontally: node's order (primary node, backup0, backup 1);
Vertically: all nodes from topology;
Z-order: count of node is placed in the specified order (e.g. node is primary nodes) for all

> Cache affinity calculation is slow with large nodes number
> ----------------------------------------------------------
>                 Key: IGNITE-3018
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3018
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>            Reporter: Semen Boikov
>            Assignee: Taras Ledkov
>            Priority: Critical
>             Fix For: 1.6
>         Attachments: 003.png, 064.png, 100.png, 128.png, 200.png, 300.png, 400.png, 500.png,
> With large number of cache server nodes (> 200)  RendezvousAffinityFunction and FairAffinityFunction
work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of milliseconds, for
FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and nodes list
sorting. As optimization we can try to cache {partion, node} MD5 hash or try another hash
function. Also several minor optimizations are possible (avoid unncecessary allocations, only
one thread local 'get', etc).

This message was sent by Atlassian JIRA

View raw message