Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of roland237@googlemail.com
 designates 209.85.220.216 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=XaiC1ueRyHU5yq6u49ijTd6eZ4cAJ0E3ODLCPVmoKbyxaW9A8D8b4+x+PZ8X9HTtoM
         6rqqT/DlBbi+vRmrMnadlSYrM/Ffpz3mgZI/Rm0BJ8IkO8dLvoFYpWmwTTKsoMQsO9V2
         ldotM0/ZfX19EOkGz6bAJlUxhSpKVa4dWQQio=
MIME-Version: 1.0
Sender: roland237@googlemail.com
In-Reply-To: <10e230a81003261435r4ec831dfy41bded43cee8ae8e@mail.gmail.com>
References: 
 <33FDEB0CE2F65F41A4CF8769247BB3668DC58A2A5C@EXVMBX016-3.exch016.msoutlookonline.net>
	 <e06563881003250856l71e15325ne7ff9bb49c65cdbb@mail.gmail.com>
	 <10e230a81003251117n681650bas7877aeb7170b6c7a@mail.gmail.com>
	 <e06563881003251126y298b7c12k5a0a1cbd35151b5d@mail.gmail.com>
	 <2545a92c1003251152h1be16180yc93e649a5ea8a91@mail.gmail.com>
	 <e06563881003251427i6880f7a4t49cd8c8f6a9f2eac@mail.gmail.com>
	 <429591151003261336j29fc6236r915e7e90d8d665c0@mail.gmail.com>
	 <10e230a81003261435r4ec831dfy41bded43cee8ae8e@mail.gmail.com>
Date: Sat, 27 Mar 2010 10:26:03 +0100
Message-ID: <429591151003270226g1034853fxa4dd097bfad5cc0a@mail.gmail.com>
Subject: Re: Ring management and load balance
From: =?ISO-8859-1?Q?Roland_H=E4nel?= <roland@haenel.me>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=0016e65b60a270821d0482c4ded7

--0016e65b60a270821d0482c4ded7
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Mike,

If you have the assumption that your rows are roughly equal in size (at
least statistcally), then you could also just take a node's total load (thi=
s
is exposed via Jmx) and divide by the amount of keys/rows on that node. Not
sure how to get the latter, but shouldn't be such a big deal to integrate i=
n
JMX if not already there.

Roland

26.03.2010 22:36 schrieb am "Mike Malone" <mike@simplegeo.com>:

2010/3/26 Roland H=E4nel <roland@haenel.me>

>
> Jonathan,
>
> I agree with your idea about a tool that could 'propose' good token
choices for op...
With the random partitioner there's no need to suggest a token. The key
space is statistically random so you should be able to just split 2^128 int=
o
equal sized segments and get fairly equal storage load. Your read / write
load could get out of whack if you have hot spots and stuff, I guess. But
for a large distributed data set I think that's unlikely.

For order preserving partitioners it's harder. We've been thinking about
this issue at SimpleGeo and were planning on implementing an algorithm that
could determine the median row key statistically without having to inspect
every key. Basically, it would pull a random sample of row keys (maybe from
the Index file?) and then determine the median of that sample. Thoughts?

Mike

--0016e65b60a270821d0482c4ded7
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<p>Mike,</p>
<p>If you have the assumption that your rows are roughly equal in size (at =
least statistcally), then you could also just take a node&#39;s total load =
(this is exposed via Jmx) and divide by the amount of keys/rows on that nod=
e. Not sure how to get the latter, but shouldn&#39;t be such a big deal to =
integrate in JMX if not already there. </p>

<p>Roland</p>
<p><blockquote type=3D"cite">26.03.2010 22:36 schrieb am &quot;Mike Malone&=
quot; &lt;<a href=3D"mailto:mike@simplegeo.com">mike@simplegeo.com</a>&gt;:=
<br><br>2010/3/26 Roland H=E4nel <span dir=3D"ltr">&lt;<a href=3D"mailto:ro=
land@haenel.me" target=3D"_blank">roland@haenel.me</a>&gt;</span><br>
<div class=3D"gmail_quote"><p><font color=3D"#500050">&gt;<br>&gt; Jonathan=
,<br>&gt;<br>&gt; I agree with your idea about a tool that could &#39;propo=
se&#39; good token choices for op...</font></p><div>With the random partiti=
oner there&#39;s no need to suggest a token. The key space is statistically=
 random so you should be able to just split 2^128 into equal sized segments=
 and get fairly equal storage load. Your read / write load could get out of=
 whack if you have hot spots and stuff, I guess. But for a large distribute=
d data set I think that&#39;s unlikely.</div>

<div><br></div><div>For order preserving partitioners it&#39;s harder. We&#=
39;ve been thinking about this issue at SimpleGeo and were planning on impl=
ementing an algorithm that could determine the median row key statistically=
 without having to inspect every key. Basically, it would pull a random sam=
ple of row keys (maybe from the Index file?) and then determine the median =
of that sample. Thoughts?</div>

<div><br></div><font color=3D"#888888"><div>Mike</div></font></div>
</blockquote></p>

--0016e65b60a270821d0482c4ded7--