Mailing-List: contact user-help@helix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@helix.apache.org
MIME-Version: 1.0
In-Reply-To: <CAC56g40mP6W=b_z-C2Pp36y2e21-XwGu-LbKc3fCgc5GjfxdqA@mail.gmail.com>
References: <CAC56g40mP6W=b_z-C2Pp36y2e21-XwGu-LbKc3fCgc5GjfxdqA@mail.gmail.com>
From: kishore g <g.kishore@gmail.com>
Date: Wed, 19 Oct 2016 12:52:20 -0700
Message-ID: <CABaj-QaKa8NRRW8g02tb9eHk1wuO-dmot_nJJ+MBKGOpVHWj_A@mail.gmail.com>
Subject: Re: Correct way to redistribute work from disconnected instances?
To: "user@helix.apache.org" <user@helix.apache.org>
Content-Type: multipart/alternative; boundary=001a113d6f10ac8e47053f3d2522
archived-at: Wed, 19 Oct 2016 19:58:16 -0000

--001a113d6f10ac8e47053f3d2522
Content-Type: text/plain; charset=UTF-8

I think this might be a corner case when partitions * replicas <
TOTAL_NUMBER_OF_NODES. Can you try with many partitions and replicas and
check if the issue still exists.


On Wed, Oct 19, 2016 at 11:53 AM, Michael Craig <mcraig@box.com> wrote:

> I've noticed that partitions/replicas assigned to disconnected instances
> are not automatically redistributed to live instances. What's the correct
> way to do this?
>
> For example, given this setup with Helix 0.6.5:
> - 1 resource
> - 2 replicas
> - LeaderStandby state model
> - FULL_AUTO rebalance mode
> - 3 nodes (N1 is Leader, N2 is Standby, N3 is just sitting)
>
> Then drop N1:
> - N2 becomes LEADER
> - Nothing happens to N3
>
> Naively, I would have expected N3 to transition from Offline to Standby,
> but that doesn't happen.
>
> I can force redistribution from GenericHelixController#onLiveInstanceChange
> by
> - dropping non-live instances from the cluster
> - calling rebalance
>
> The instance dropping seems pretty unsafe! Is there a better way?
>

--001a113d6f10ac8e47053f3d2522
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I think this might be a corner case when partitions * repl=
icas &lt; TOTAL_NUMBER_OF_NODES. Can you try with many partitions and repli=
cas and check if the issue still exists.<div><br></div><div><br></div></div=
><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Oct 19, =
2016 at 11:53 AM, Michael Craig <span dir=3D"ltr">&lt;<a href=3D"mailto:mcr=
aig@box.com" target=3D"_blank">mcraig@box.com</a>&gt;</span> wrote:<br><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><div dir=3D"ltr"><div>I&#39;ve noticed that part=
itions/replicas assigned to disconnected instances are not automatically re=
distributed to live instances. What&#39;s the correct way to do this?</div>=
<div><br></div>For example, given this setup with Helix 0.6.5:<div>- 1 reso=
urce<br></div><div>- 2 replicas</div><div>- LeaderStandby state model</div>=
<div>- FULL_AUTO rebalance mode</div><div>- 3 nodes (N1 is Leader, N2 is St=
andby, N3 is just sitting)</div><div><br></div><div>Then drop N1:</div><div=
>- N2 becomes LEADER</div><div>- Nothing happens to N3</div><div><br></div>=
<div>Naively, I would have expected N3 to transition from Offline to Standb=
y, but that doesn&#39;t happen.</div><div><br></div><div>I can force redist=
ribution from GenericHelixController#onLiveI<wbr>nstanceChange by</div><div=
>- dropping non-live instances from the cluster</div><div>- calling rebalan=
ce</div><div><br></div><div>The instance dropping seems pretty unsafe! Is t=
here a better way?</div></div>
</blockquote></div><br></div>

--001a113d6f10ac8e47053f3d2522--