Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of owenzhang1990@gmail.com
 designates 74.125.83.53 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAL35Oi0Sr_NqUuJgOnqq=SAo7ByY0G4c+Lqsg_PsCnPjpmUnrQ@mail.gmail.com>
References: 
 <CA+BDQ7zf+fOdq+BdA8_C2cdMNJh6f8i5pS7iFaOMjO7OriN8HA@mail.gmail.com>
	<CAL35Oi2E4aSTUAB5i6hwvaGJCpcNgk6CU=wHgM_XQhEy7s6H=w@mail.gmail.com>
	<CABT57mZW-Fx65FkpWRFdeo-RK6Pp2ZaMBbSGP6nxNQE2q9hGrg@mail.gmail.com>
	<CAAafH9T5W5Gr1kgKsK-mPZOhHKeAdard4jVLzEaKj9SWo6n5rA@mail.gmail.com>
	<CABT57mbmJ_8GKvXPT+-Tb1g06kT3RoxN4nG-3gYwYw_CyHn4Eg@mail.gmail.com>
	<CAL35Oi0Sr_NqUuJgOnqq=SAo7ByY0G4c+Lqsg_PsCnPjpmUnrQ@mail.gmail.com>
Date: Thu, 10 Jan 2013 01:18:27 +0800
Message-ID: 
 <CABT57mYXNqpCAN2Rz-uPTEj3D==Y89=W0AsRuqCsCS4WN41GPA@mail.gmail.com>
Subject: Re: distribution of token ranges with virtual nodes
From: Manu Zhang <owenzhang1990@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=047d7b62212420580c04d2de404a

--047d7b62212420580c04d2de404a
Content-Type: text/plain; charset=ISO-8859-1

Is cassandra-shuffle command in the trunk? Or it is only included in the
Debian package? I don't find it in the trunk.


On Sat, Nov 3, 2012 at 2:18 AM, Eric Evans <eevans@acunu.com> wrote:

> On Fri, Nov 2, 2012 at 12:38 AM, Manu Zhang <owenzhang1990@gmail.com>
> wrote:
> >> It splits into a contiguous range, because truly upgrading to vnode
> >> functionality is another step.
> >
> > That confuses me. As I understand it, there is no point in having 256
> tokens
> > on same node if I don't commit the shuffle
>
> This isn't exactly true.  By-partition operations (think repair,
> streaming, etc) will be more reliable in the sense that if they fail
> and need to be restarted, there is less that is lost/needs redoing.
> Also, if all you did was migrate from 1-token-per-node to 256
> contiguous tokens per node, normal topology changes (bootstrapping new
> nodes, decommissioning old ones), would gradually work to redistribute
> the partitions.  And, from a topology perspective, splitting the one
> partition into many contiguous partition is a no-op; it's safe to do
> and there is no cost to speak of from a computational or IO
> perspective.
>
> On the other hand, shuffling requires moving tokens around the
> cluster.  If you completely randomize placement, it follows that you
> will need to relocate all of the clusters data, so it's quite costly.
> It's also precedent setting, and not thoroughly tested yet.
>
> --
> Eric Evans
> Acunu | http://www.acunu.com | @acunu
>

--047d7b62212420580c04d2de404a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Is cassandra-shuffle command in the trunk? Or it is only i=
ncluded in the Debian package? I don&#39;t find it in the trunk.</div><div =
class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Sat, Nov 3, 201=
2 at 2:18 AM, Eric Evans <span dir=3D"ltr">&lt;<a href=3D"mailto:eevans@acu=
nu.com" target=3D"_blank">eevans@acunu.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"im">On Fri, Nov 2, 2012 at 12:=
38 AM, Manu Zhang &lt;<a href=3D"mailto:owenzhang1990@gmail.com">owenzhang1=
990@gmail.com</a>&gt; wrote:<br>

&gt;&gt; It splits into a contiguous range, because truly upgrading to vnod=
e<br>
&gt;&gt; functionality is another step.<br>
&gt;<br>
&gt; That confuses me. As I understand it, there is no point in having 256 =
tokens<br>
&gt; on same node if I don&#39;t commit the shuffle<br>
<br>
</div>This isn&#39;t exactly true. =A0By-partition operations (think repair=
,<br>
streaming, etc) will be more reliable in the sense that if they fail<br>
and need to be restarted, there is less that is lost/needs redoing.<br>
Also, if all you did was migrate from 1-token-per-node to 256<br>
contiguous tokens per node, normal topology changes (bootstrapping new<br>
nodes, decommissioning old ones), would gradually work to redistribute<br>
the partitions. =A0And, from a topology perspective, splitting the one<br>
partition into many contiguous partition is a no-op; it&#39;s safe to do<br=
>
and there is no cost to speak of from a computational or IO<br>
perspective.<br>
<br>
On the other hand, shuffling requires moving tokens around the<br>
cluster. =A0If you completely randomize placement, it follows that you<br>
will need to relocate all of the clusters data, so it&#39;s quite costly.<b=
r>
It&#39;s also precedent setting, and not thoroughly tested yet.<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
--<br>
Eric Evans<br>
Acunu | <a href=3D"http://www.acunu.com" target=3D"_blank">http://www.acunu=
.com</a> | @acunu<br>
</div></div></blockquote></div><br></div>

--047d7b62212420580c04d2de404a--