From user-return-62452-archive-asf-public=cust-asf.ponee.io@cassandra.apache.org Tue Oct 2 10:28:26 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id CC2EE180638 for ; Tue, 2 Oct 2018 10:28:25 +0200 (CEST) Received: (qmail 28686 invoked by uid 500); 2 Oct 2018 08:28:23 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 28675 invoked by uid 99); 2 Oct 2018 08:28:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Oct 2018 08:28:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 503CCC6002 for ; Tue, 2 Oct 2018 08:28:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.151 X-Spam-Level: *** X-Spam-Status: No, score=3.151 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_REPLY=1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 7teE7c5MFUli for ; Tue, 2 Oct 2018 08:28:22 +0000 (UTC) Received: from mail-yw1-f46.google.com (mail-yw1-f46.google.com [209.85.161.46]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 31B265F32F for ; Tue, 2 Oct 2018 08:28:22 +0000 (UTC) Received: by mail-yw1-f46.google.com with SMTP id l79-v6so420702ywc.7 for ; Tue, 02 Oct 2018 01:28:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=pAAaJCfmoIOtWvDxfO3S0tYdcJCXYeWAv5m6rv5AGvA=; b=VN8P/HKRZYrJVwNGj9b/gwDGD21Qo7N9MWd9QXa2I7qpnL2K0lx6xXgFrrpxXv6Qi5 GB0ZMMrO58/4i6wWY2SPI8NXc5D7iTUFjUlrNtBm5dG4cxVSsrt+HXvjJDyIKOhVIMIq iSBZfLVbimkd7gjPLfmyWrQ51qJiVdbJ0wCKHRlxU3maEB321cKhKouDrQ29gE2Rm0IS yd5fBb+blk8bTQm6uyUAbntTC9m8LT1chVNL7BZBt+MEovhPi80P7Cy+bdsQc0EXdRF3 0YfL1mMM6Ie4hmEoZQOjykdql8ZJ6GJKdJxtc/9/JUlj0sBxbhLBfY3jugJ3Kmr2quVQ xUcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=pAAaJCfmoIOtWvDxfO3S0tYdcJCXYeWAv5m6rv5AGvA=; b=Sj6rxMOTzTbrRNKyko6BTVGnEcHaW2X8hPrWXvz4guTvp1NdvSdT82aoIrdt5w/QzR 87UTwaHo9i6uUvPnCm4VQUzy2ZZ8tgYkO5ICntt2LUGGQ4sd/jch0ctaqrG96Q2Eqwq5 CPRq+4gx4iwvSBO6tARAbLyAkpe91vrC31wrDdLEqo3QtGI7myzNZfkUvdjIl7aRgUu5 dHKaYCSadFfkVG6nbNkV3bKBB4aJJ7lZ8/VMxTVUepn7xt92VK9TwEootfyCA0TmY97h GjGtkjb2jCP7gkflnIPdOAWDKSlAcmthhv4zKQyiywUsvLcBO8CYiaCvJGT9P0+8FSor YCbw== X-Gm-Message-State: ABuFfoiwreMPTUU9SPZNDeAXd0de/NtPk3aLT66OQKp+D3D2HlJnYhQa sMHyYWxpY9I+qOtlZOx0PrV1qFgmE88npAJRjbCBTBWb X-Google-Smtp-Source: ACcGV61APZwbh+4OXjZBJlDJB5o4BImGRtvsi22ALU9YwQ68hlJnkF1btYY931f5a+7A1ha6vqI90++T74GljzDeKMA= X-Received: by 2002:a81:a11:: with SMTP id 17-v6mr7717432ywk.238.1538468896136; Tue, 02 Oct 2018 01:28:16 -0700 (PDT) MIME-Version: 1.0 References: <1662e0e64bd.125be70e941930.7215874111596348455@zoho.com> <1662f227495.c503490a45862.6653275483034403400@zoho.com> <1662f3474ad.bf2a8d5046144.4729157283643333394@zoho.com> <166337908a8.104edcdb466026.8375725610568926021@zoho.com> In-Reply-To: <166337908a8.104edcdb466026.8375725610568926021@zoho.com> From: Varun Barala Date: Tue, 2 Oct 2018 16:28:04 +0800 Message-ID: Subject: Re: Re: Re: how to configure the Token Allocation Algorithm To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary="0000000000000d61b505773ab45e" --0000000000000d61b505773ab45e Content-Type: text/plain; charset="UTF-8" Hi, Managing `initial_token` by yourself will give you more control over scale-in and scale-out. Let's say you have three node cluster with `num_token: 1` And your initial range looks like:- Datacenter: datacenter1 ========== Address Rack Status State Load Owns Token 3074457345618258602 127.0.0.1 rack1 Up Normal 98.96 KiB 66.67% -9223372036854775808 127.0.0.2 rack1 Up Normal 98.96 KiB 66.67% -3074457345618258603 127.0.0.3 rack1 Up Normal 98.96 KiB 66.67% 3074457345618258602 Now let's say you want to scale out the cluster to twice the current throughput(means you are adding 3 more nodes) If you are using AWS EBS volumes then you can use the same volumes and spin three more nodes by selecting midpoints of existing ranges which means your new nodes are already having data. Once you have mounted volumes on your new nodes:- * You need to delete every system table except schema related tables. * You need to generate system/local table by yourself which has `Bootstrap state` as completed and schema-version same as other existing nodes. * You need to remove extra data on all the machines using cleanup commands This is how you can scale out Cassandra cluster in the minutes. In case you want to add nodes one by one then you need to write some small tool which will always figure out the bigger range in the existing cluster and will split it into the half. However, I never tested it thoroughly but this should work conceptually. So here we are taking advantage of the fact that we have volumes(data) for the new node beforehand so we no need to bootstrap them. Thanks & Regards, Varun Barala On Tue, Oct 2, 2018 at 2:31 PM onmstester onmstester wrote: > > > Sent using Zoho Mail > > > ---- On Mon, 01 Oct 2018 18:36:03 +0330 *Alain RODRIGUEZ > >* wrote ---- > > Hello again :), > > I thought a little bit more about this question, and I was actually > wondering if something like this would work: > > Imagine 3 node cluster, and create them using: > For the 3 nodes: `num_token: 4` > Node 1: `intial_token: -9223372036854775808, -4611686018427387905, -2, > 4611686018427387901` > Node 2: `intial_token: -7686143364045646507, -3074457345618258604, > 1537228672809129299, 6148914691236517202` > Node 3: `intial_token: -6148914691236517206, -1537228672809129303, > 3074457345618258600, 7686143364045646503` > > If you know the initial size of your cluster, you can calculate the total > number of tokens: number of nodes * vnodes and use the formula/python > code above to get the tokens. Then use the first token for the first node, > move to the second node, use the second token and repeat. In my case there > is a total of 12 tokens (3 nodes, 4 tokens each) > ``` > >>> number_of_tokens = 12 > >>> [str(((2**64 / number_of_tokens) * i) - 2**63) for i in > range(number_of_tokens)] > ['-9223372036854775808', '-7686143364045646507', '-6148914691236517206', > '-4611686018427387905', '-3074457345618258604', '-1537228672809129303', > '-2', '1537228672809129299', '3074457345618258600', '4611686018427387901', > '6148914691236517202', '7686143364045646503'] > ``` > > > Using manual initial_token (your idea), how could i add a new node to a > long running cluster (the procedure)? > > --0000000000000d61b505773ab45e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Managing `initial_token` by yo= urself will give you more control over scale-in and scale-out.
Let's= say you have three node cluster with `num_token: 1`

And your initia= l range looks like:-

Datacenter: datacenter1
=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D
Address =C2=A0 =C2=A0Rack =C2=A0 =C2=A0 = =C2=A0 =C2=A0Status State =C2=A0 Load =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0Owns =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Token =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A030= 74457345618258602 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0
127.0.0.1 =C2=A0rack1 =C2=A0 = =C2=A0 =C2=A0 Up =C2=A0 =C2=A0 Normal =C2=A098.96 KiB =C2=A0 =C2=A0 =C2=A0 = 66.67% =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-9223372036854775808= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0
127.0.0.2 =C2=A0rack1 =C2=A0 =C2=A0 =C2=A0 Up =C2=A0 = =C2=A0 Normal =C2=A098.96 KiB =C2=A0 =C2=A0 =C2=A0 66.67% =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-3074457345618258603 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
12= 7.0.0.3 =C2=A0rack1 =C2=A0 =C2=A0 =C2=A0 Up =C2=A0 =C2=A0 Normal =C2=A098.9= 6 KiB =C2=A0 =C2=A0 =C2=A0 66.67% =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A03074457345618258602 =C2=A0

Now let's say you want to scal= e out the cluster to twice the current throughput(means you are adding 3 mo= re nodes)

If you are using AWS EBS volumes then you can use the same= volumes and spin three more nodes by selecting midpoints of existing range= s which means your new nodes are already having data.
Once you have moun= ted volumes on your new nodes:-
* You need to delete every system table = except schema related tables.
* You need to generate system/local table = by yourself which has `Bootstrap state` as completed and schema-version sam= e as other existing nodes.
* You need to remove extra data on all the ma= chines using cleanup commands

This is how you can scale out Cassandr= a cluster in the minutes. In case you want to add nodes one by one then you= need to write some small tool which will always figure out the bigger rang= e in the existing cluster and will split it into the half.

How= ever, I never tested it thoroughly but this should work conceptually. So he= re we are taking advantage of the fact that we have volumes(data) for the= =C2=A0new node beforehand so we no need to bootstrap them.

Thanks &a= mp; Regards,
Varun Barala

On Tue, Oct 2, 2018 at 2:31 PM onmstester onmstester <onmstester@zoho.com> wrote:


Sent using= Zoho Mail



---- On Mon, 01 Oct 2018 18:36:03 +0330=C2=A0Alain= RODRIGUEZ <arod= rime@gmail.com> wrote ----

Hello= again :),

I thought a= little bit more about this question, and I was actually wondering=C2=A0if = something like this would work:

Imagine 3 node cluster, and create them using:
For the 3 nodes: `num_token: 4`
Node 1: `intial_token: -9223372036854775808, -4611686018427387905,= -2, 4611686018427387901`
Node 2: `intial_token: = -7686143364045646507, -3074457345618258604, 1537228672809129299, 6148914691= 236517202`
Node 3: `intial_token: -61489146912365= 17206, -1537228672809129303, 3074457345618258600, 7686143364045646503`
<= /div>

=C2=A0If you know the initial size of = your cluster, you can calculate the total number of tokens:=C2=A0number=C2=A0of nodes *=C2=A0vnodes=C2=A0and use the formula/python code above to get the tokens. Then use = the first token for the first node, move to the second node, use the second= token and repeat. In my case there is a total of 12 tokens (3 nodes, 4 tok= ens each)
```
>>&= gt; number_of_tokens =3D 12
>>> [str(((2= **64 / number_of_tokens) * i) - 2**63) for i in range(number_of_tokens)]
['-9223372036854775808', '-768614336404= 5646507', '-6148914691236517206', '-4611686018427387905'= ;, '-3074457345618258604', '-1537228672809129303', '-2&= #39;, '1537228672809129299', '3074457345618258600', '46= 11686018427387901', '6148914691236517202', '768614336404564= 6503']
```

=

Using manual initial_token (your idea), how could i add= a new node to a long running cluster (the procedure)?

--0000000000000d61b505773ab45e--