ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: Prohibit stateful affinity (FairAffinityFunction)
Date Mon, 10 Apr 2017 09:42:06 GMT
Why wouldn't we have useBalancer always enabled?

Sergi

2017-04-10 12:31 GMT+03:00 Taras Ledkov <tledkov@gridgain.com>:

> Folks,
>
> I worked on issue https://issues.apache.org/jira/browse/IGNITE-3018 that
> is related to performance of Rendezvous AF.
>
> But Wang/Jenkins hash integer hash distribution is worse then MD5. So, i
> try to use simple partition balancer close
> to Fair AF for Rendezvous AF.
>
> Take a look at the heatmaps of distributions at the issue. e.g.:
> - Compare of current Rendezvous AF and new Rendezvous AF based of
> Wang/Jenkins hash: https://issues.apache.org/jira
> /secure/attachment/12858701/004.png
> - Compare of current Rendezvous AF and new Rendezvous AF based of
> Wang/Jenkins hash with partition balancer: https://issues.apache.org/jira
> /secure/attachment/12858690/balanced.004.png
>
> When the balancer is enabled the distribution of partitions by nodes looks
> like close to even distribution
> but in this case there is not guarantee that a partition doesn't move from
> one node to another
> when node leave topology.
> It is not guarantee but we try to minimize it because sorted array of
> nodes is used (like in for pure-Rendezvous AF).
>
> I think we can use new fast Rendezvous AF and use 'useBalancer' flag
> instead of Fair AF.
>
> On 09.04.2017 14:12, Valentin Kulichenko wrote:
>
>> What is the replacement for FairAffinityFunction?
>>
>> Generally I agree. If FairAffinityFunction can't be changed to provide
>> consistent mapping, it should be dropped.
>>
>> -Val
>>
>> On Sun, Apr 9, 2017 at 3:50 AM, Sergi Vladykin <sergi.vladykin@gmail.com
>> <mailto:sergi.vladykin@gmail.com>> wrote:
>>
>>     Guys,
>>
>>     It appeared that our FairAffinityFunction can assign the same
>>     partitions to
>>     different nodes for different caches.
>>
>>     It basically means that there is no collocation between the caches
>>     at all
>>     even if they have the same affinity.
>>
>>     As a result all SQL joins will not work (even collocated ones), other
>>     operations that rely on cache collocation will be either broken or
>>     work
>>     slower, than expected.
>>
>>     All this stuff is really non-obvious. And I see no reason why we
>>     should
>>     allow that. I suggest to prohibit this behavior and drop
>>     FairAffinityFunction before 2.0. We have to clearly document that
>>     the same
>>     affinity function must provide the same partition assignments for
>>     all the
>>     caches.
>>
>>     Also I know that Taras Ledkov was working on a decent stateless
>>     replacement
>>     for FairAffinity, so we should not loose anything here.
>>
>>     Thoughts?
>>
>>     Sergi
>>
>>
>>
> --
> Taras Ledkov
> Mail-To: tledkov@gridgain.com
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message