ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taras Ledkov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
Date Wed, 27 Apr 2016 09:00:24 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259816#comment-15259816
] 

Taras Ledkov commented on IGNITE-3018:
--------------------------------------

Use 32/64 bits [Wang/Jenkins hash|https://gist.github.com/badboy/6267743] and build-in hashCode()
 for ClusterNode hash object for instead of MD5.
Approximates times of call the RendezvousAffinityFunction.assignPartitions. The test replaces
the last node 30 times for 10 instances of affinity function (total 300 times).

64 bits implementation packs partition number and node.hashCode() to long and [mixes|https://gist.github.com/badboy/6267743#64-bit-mix-functions]
the long key:

Test 100 nodes. Old: 78 ms +/- 11.386 ms; New: 17 ms +/- 7.483 ms;
Test 200 nodes. Old: 153 ms +/- 13.065 ms; New: 36 ms +/- 9.309 ms;
Test 300 nodes. Old: 233 ms +/- 12.855 ms; New: 57 ms +/- 10.343 ms;
Test 400 nodes. Old: 314 ms +/- 11.572 ms; New: 80 ms +/- 10.349 ms;
Test 500 nodes. Old: 395 ms +/- 13.838 ms; New: 101 ms +/- 7.118 ms;
Test 600 nodes. Old: 476 ms +/- 15.727 ms; New: 123 ms +/- 10.732 ms;


32 bits implementation hashes partition number and node.hashCode() to int and [mixes|https://gist.github.com/badboy/6267743#using-multiplication-for-hashing])
the integer key:

Test 100 nodes. Old: 74 ms +/- 10.611 ms; New: 17 ms +/- 7.490 ms;
Test 200 nodes. Old: 149 ms +/- 13.397 ms; New: 37 ms +/- 9.592 ms;
Test 300 nodes. Old: 227 ms +/- 12.745 ms; New: 57 ms +/- 10.110 ms;
Test 400 nodes. Old: 307 ms +/- 13.166 ms; New: 81 ms +/- 10.890 ms;
Test 500 nodes. Old: 385 ms +/- 16.531 ms; New: 103 ms +/- 9.490 ms;
Test 600 nodes. Old: 467 ms +/- 18.128 ms; New: 128 ms +/- 12.157 ms;


> Cache affinity calculation is slow with large nodes number
> ----------------------------------------------------------
>
>                 Key: IGNITE-3018
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3018
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>            Reporter: Semen Boikov
>            Assignee: Taras Ledkov
>            Priority: Critical
>             Fix For: 1.6
>
>
> With large number of cache server nodes (> 200)  RendezvousAffinityFunction and FairAffinityFunction
work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of milliseconds, for
FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and nodes list
sorting. As optimization we can try to cache {partion, node} MD5 hash or try another hash
function. Also several minor optimizations are possible (avoid unncecessary allocations, only
one thread local 'get', etc).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message