ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: Prohibit stateful affinity (FairAffinityFunction)
Date Mon, 10 Apr 2017 09:43:25 GMT
We should have it enabled by default.

--Yakov

2017-04-10 12:42 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:

> Why wouldn't we have useBalancer always enabled?
>
> Sergi
>
> 2017-04-10 12:31 GMT+03:00 Taras Ledkov <tledkov@gridgain.com>:
>
> > Folks,
> >
> > I worked on issue https://issues.apache.org/jira/browse/IGNITE-3018 that
> > is related to performance of Rendezvous AF.
> >
> > But Wang/Jenkins hash integer hash distribution is worse then MD5. So, i
> > try to use simple partition balancer close
> > to Fair AF for Rendezvous AF.
> >
> > Take a look at the heatmaps of distributions at the issue. e.g.:
> > - Compare of current Rendezvous AF and new Rendezvous AF based of
> > Wang/Jenkins hash: https://issues.apache.org/jira
> > /secure/attachment/12858701/004.png
> > - Compare of current Rendezvous AF and new Rendezvous AF based of
> > Wang/Jenkins hash with partition balancer:
> https://issues.apache.org/jira
> > /secure/attachment/12858690/balanced.004.png
> >
> > When the balancer is enabled the distribution of partitions by nodes
> looks
> > like close to even distribution
> > but in this case there is not guarantee that a partition doesn't move
> from
> > one node to another
> > when node leave topology.
> > It is not guarantee but we try to minimize it because sorted array of
> > nodes is used (like in for pure-Rendezvous AF).
> >
> > I think we can use new fast Rendezvous AF and use 'useBalancer' flag
> > instead of Fair AF.
> >
> > On 09.04.2017 14:12, Valentin Kulichenko wrote:
> >
> >> What is the replacement for FairAffinityFunction?
> >>
> >> Generally I agree. If FairAffinityFunction can't be changed to provide
> >> consistent mapping, it should be dropped.
> >>
> >> -Val
> >>
> >> On Sun, Apr 9, 2017 at 3:50 AM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> >> <mailto:sergi.vladykin@gmail.com>> wrote:
> >>
> >>     Guys,
> >>
> >>     It appeared that our FairAffinityFunction can assign the same
> >>     partitions to
> >>     different nodes for different caches.
> >>
> >>     It basically means that there is no collocation between the caches
> >>     at all
> >>     even if they have the same affinity.
> >>
> >>     As a result all SQL joins will not work (even collocated ones),
> other
> >>     operations that rely on cache collocation will be either broken or
> >>     work
> >>     slower, than expected.
> >>
> >>     All this stuff is really non-obvious. And I see no reason why we
> >>     should
> >>     allow that. I suggest to prohibit this behavior and drop
> >>     FairAffinityFunction before 2.0. We have to clearly document that
> >>     the same
> >>     affinity function must provide the same partition assignments for
> >>     all the
> >>     caches.
> >>
> >>     Also I know that Taras Ledkov was working on a decent stateless
> >>     replacement
> >>     for FairAffinity, so we should not loose anything here.
> >>
> >>     Thoughts?
> >>
> >>     Sergi
> >>
> >>
> >>
> > --
> > Taras Ledkov
> > Mail-To: tledkov@gridgain.com
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message