Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <1469425318078-6502.post@n6.nabble.com>
References: <1469425318078-6502.post@n6.nabble.com>
From: Vladislav Pyatkov <vldpyatkov@gmail.com>
Date: Mon, 25 Jul 2016 09:38:25 +0300
Message-ID: <CADez4_UcXWE8OG2_6ur5je_FhYaQjmnS1Cn-ofuKiHbSaoLVBQ@mail.gmail.com>
Subject: Re: How to control the data unbalance among all the server nodes?
To: user@ignite.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0b34fe105ee305387008dd
archived-at: Mon, 25 Jul 2016 06:38:38 -0000

--94eb2c0b34fe105ee305387008dd
Content-Type: text/plain; charset=UTF-8

Hello Jason,

1) Yes, data (cache entry) distributed in partition, depends on hash code
of key (key.hashCode() % parts).

2) The basis for the rebalancing of partitions are leave node from cluster
or join a node to cluster. RendezvousAffinityFunction does so that each
node has roughly the same count of partitions. Also
(RendezvousAffinityFunction)  partition distribution as less as possible
difference than the previous.

Thus we can assume (in case the hash code has uniform distribution) that
each node has about a different part of the data.

On Mon, Jul 25, 2016 at 8:41 AM, Jason <fqyang@outlook.com> wrote:

> hi Ignite team,
>
> 1. Ignite does the data balance based on the cache partitions' balance not
> the data, right? E.g. if there's data skew in some keys, how to handle?
>
> 2. RendezvousAffinityFunction cannot ensure all the nodes have the
> completely same partitions and this should be an advantage to reduce the
> impact of network fluctuations, such as Node leave/join, but how to control
> the unbalance rate of all the nodes, say partition # in the node with max
> partitions / that of min? If the max node reaches the memory limit, will it
> do re-balance? Is there any config to control this, say 10%, 20%?
>
> Thanks,
> -Jason
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/How-to-control-the-data-unbalance-among-all-the-server-nodes-tp6502.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>


-- 
Vladislav Pyatkov

--94eb2c0b34fe105ee305387008dd
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hello=C2=A0<span style=3D"font-size:12.8px">Jason,</s=
pan></div><div><br></div><div>1) Yes, data (cache entry) distributed in par=
tition, depends on hash code of key (key.hashCode() % parts).</div><div><br=
></div><div>2) The basis for the rebalancing of partitions are leave node f=
rom cluster or join a node to cluster. RendezvousAffinityFunction does so t=
hat each node has roughly the same count of partitions. Also (RendezvousAff=
inityFunction) =C2=A0partition distribution as less as possible difference =
than the previous.</div><div><br></div><div>Thus we can assume (in case the=
 hash code has uniform distribution) that each node has about a different p=
art of the data.</div></div><div class=3D"gmail_extra"><br><div class=3D"gm=
ail_quote">On Mon, Jul 25, 2016 at 8:41 AM, Jason <span dir=3D"ltr">&lt;<a =
href=3D"mailto:fqyang@outlook.com" target=3D"_blank">fqyang@outlook.com</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">hi Ignite team,<br>
<br>
1. Ignite does the data balance based on the cache partitions&#39; balance =
not<br>
the data, right? E.g. if there&#39;s data skew in some keys, how to handle?=
<br>
<br>
2. RendezvousAffinityFunction cannot ensure all the nodes have the<br>
completely same partitions and this should be an advantage to reduce the<br=
>
impact of network fluctuations, such as Node leave/join, but how to control=
<br>
the unbalance rate of all the nodes, say partition # in the node with max<b=
r>
partitions / that of min? If the max node reaches the memory limit, will it=
<br>
do re-balance? Is there any config to control this, say 10%, 20%?<br>
<br>
Thanks,<br>
-Jason<br>
<br>
<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-ignite-users.70518.x=
6.nabble.com/How-to-control-the-data-unbalance-among-all-the-server-nodes-t=
p6502.html" rel=3D"noreferrer" target=3D"_blank">http://apache-ignite-users=
.70518.x6.nabble.com/How-to-control-the-data-unbalance-among-all-the-server=
-nodes-tp6502.html</a><br>
Sent from the Apache Ignite Users mailing list archive at Nabble.com.<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature" data-smartmail=3D"gmail_signature">Vladislav Pyatkov</=
div>
</div>

--94eb2c0b34fe105ee305387008dd--