Mailing-List: contact user-help@kudu.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@kudu.apache.org
MIME-Version: 1.0
In-Reply-To: <CAGpTDNcgBwL+ygsA6=4rQk0t=_eN8ob+hGc1TLGK+7x+gP4Tgg@mail.gmail.com>
References: <CACXRouw3tPwz+6FHPtc3jqMxQx_X7ioO0+24O9EWCTCWXVFXow@mail.gmail.com>
 <CAGpTDNcgBwL+ygsA6=4rQk0t=_eN8ob+hGc1TLGK+7x+gP4Tgg@mail.gmail.com>
From: =?UTF-8?Q?Alexandre_Fouch=C3=A9?= <afouche@onfocus.io>
Date: Thu, 9 Mar 2017 20:05:24 +0100
Message-ID: <CACXRouw9Fvfx_GsW=yMCONRD2+GVUvsCOcAvGw_Pk4Vt1YLAaQ@mail.gmail.com>
Subject: Re: How to reuse tablet server UUID, or removing old one
To: user@kudu.apache.org
Content-Type: multipart/alternative; boundary=001a114c98b46a8b08054a50ed5a
archived-at: Thu, 09 Mar 2017 19:05:37 -0000

--001a114c98b46a8b08054a50ed5a
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Oh ok, so since i need to do replace partitionning on all other nodes, i
suppose tabletserver5 will get populated when i delete other tabletservers
one by one.

And indeed, now i see from the webUI, that all my tablets still have 3
replicas, so Kudu must have ensured a replication of 3 when tabletserver5
was considered dead. (i wonder how much time after though)

Yet, is it possible to have the masters forget about the dead tablet server
UUID, so that it does not show up anymore in the webUI ? Or will it
disapperar after a while or a Kudu restart maybe ?


2017-03-09 17:48 GMT+01:00 Jean-Daniel Cryans <jdcryans@apache.org>:

> Hi Alexandre,
>
> Tablet replicas are not tied to a UUID, so removing or reusing one
> wouldn't achieve what you want. The main thing missing here is that Kudu
> doesn't do tablet re-balancing at runtime, so tabletserver5 will get
> tablets the next time a node dies or if you create new tables.
>
> Obviously that's something we'd like to address but nobody has come aroun=
d
> to doing it so far.
>
> Sinc=C3=A8rement,
>
> J-D
>
> On Thu, Mar 9, 2017 at 6:34 AM, Alexandre Fouch=C3=A9 <afouche@onfocus.io=
>
> wrote:
>
>> Hi all
>>
>> I have searched and searched, but could not find how to tell that a
>> previous tablet server uuid is to be removed
>> I had to replace disks on tabletserver5, so i deleted all data, since
>> there were replicas on other servers. Now that i restarted Kudu on
>> tabletserver5 with empty data, it initialised fine, but since (i saw
>> afterwards on webUI) it has a new UUID, it is recognised as a new server=
,
>> and it does not resync its tablets replcas with other tablet servers
>> (edited). It has the same hostname as before but different UUID
>>
>> And in WebUI, i see the same tabletserver5 with the previous UUID marked
>> as =E2=80=98dead'
>>
>> How can i tel Kudu to completely remove the dead tabletserver5 UUID and
>> populate the new tabletserver5 UUID instead ?
>>
>> the `kudu` command line tool does not seem to allow to delete a tablet
>> server UUID, or decommission
>> so how ?
>>
>> Or other way, how can i recreate an empty Kudu tablet server reusing my
>> old UUID ?
>>
>
>

--001a114c98b46a8b08054a50ed5a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>Oh ok, so since i need to do replace partitionni=
ng on all other nodes, i suppose tabletserver5 will get populated when i de=
lete other tabletservers one by one.<br><br></div>And indeed, now i see fro=
m the webUI, that all my tablets still have 3 replicas, so Kudu must have e=
nsured a replication of 3 when tabletserver5 was considered dead. (i wonder=
 how much time after though)<br><br></div>Yet, is it possible to have the m=
asters forget about the dead tablet server UUID, so that it does not show u=
p anymore in the webUI ? Or will it disapperar after a while or a Kudu rest=
art maybe ?<br><br></div><div class=3D"gmail_extra"><br><div class=3D"gmail=
_quote">2017-03-09 17:48 GMT+01:00 Jean-Daniel Cryans <span dir=3D"ltr">&lt=
;<a href=3D"mailto:jdcryans@apache.org" target=3D"_blank">jdcryans@apache.o=
rg</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi Al=
exandre,<div><br></div><div>Tablet replicas are not tied to a UUID, so remo=
ving or reusing one wouldn&#39;t achieve what you want. The main thing miss=
ing here is that Kudu doesn&#39;t do tablet re-balancing at runtime, so tab=
letserver5 will get tablets the next time a node dies or if you create new =
tables.</div><div><br></div><div>Obviously that&#39;s something we&#39;d li=
ke to address but nobody has come around to doing it so far.</div><div><br>=
</div><div>Sinc=C3=A8rement,</div><div><br></div><div>J-D</div></div><div c=
lass=3D"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Thu, Mar 9, 2017 at 6:34 AM, Alexandre Fouch=C3=A9 <spa=
n dir=3D"ltr">&lt;<a href=3D"mailto:afouche@onfocus.io" target=3D"_blank">a=
fouche@onfocus.io</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote=
" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><=
div dir=3D"ltr">Hi all<br><br>I have searched and searched, but could not f=
ind how to tell that a previous tablet server uuid is to be removed<br>I ha=
d to replace disks on tabletserver5, so i deleted all data, since there wer=
e replicas on other servers. Now that i restarted Kudu on tabletserver5 wit=
h empty data, it initialised fine, but since (i saw afterwards on webUI) it=
 has a new UUID, it is recognised as a new server, and it does not resync i=
ts tablets replcas with other tablet servers (edited). It has the same host=
name as before but different UUID<br><br>And in WebUI, i see the same table=
tserver5 with the previous UUID marked as =E2=80=98dead&#39;<br><br>How can=
 i tel Kudu to completely remove the dead tabletserver5 UUID and populate t=
he new tabletserver5 UUID instead ?<br><br>the `kudu` command line tool doe=
s not seem to allow to delete a tablet server UUID, or decommission<br>so h=
ow ?<br><br>Or other way, how can i recreate an empty Kudu tablet server re=
using my old UUID ?<br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a114c98b46a8b08054a50ed5a--