Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
From: Peter Hsu <peter@motivecast.com>
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: multipart/alternative; boundary=Apple-Mail-83-236363569
Subject: Re: Order Preserving Partitioner
Date: Wed, 26 May 2010 10:51:46 -0700
In-Reply-To: <AANLkTikDTO_6VZVyTjwxR4hUx42Npszv9RBCQ_wfuzXs@mail.gmail.com>
To: user@cassandra.apache.org
References: <AANLkTinTZGewQgDCIXoW4bSgKXYWlMEIPaxT-lrhBXbV@mail.gmail.com>
 <AANLkTikDTO_6VZVyTjwxR4hUx42Npszv9RBCQ_wfuzXs@mail.gmail.com>
Message-Id: <A605AF68-D2BB-4804-9EA4-FB10FD509CA8@motivecast.com>


--Apple-Mail-83-236363569
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Correct me if I'm wrong here.  Even though you can get your results with =
Random Partitioner, it's a lot less efficient if you're going across =
different machines to get your results.  If you're doing a lot of range =
queries, it makes sense to have things ordered sequentially so that if =
you do need to go to disk, the reads will be faster, rather than lots of =
random reads across your system.

It's also my understanding that if you go with the OPP, you could hash =
your key yourself using md5 or sha-1 to effectively get random =
partitioning.  So it's a bit of a pain, but not impossible to do a split =
between OPP and RP for your different columnfamily/keyspaces.

On May 26, 2010, at 2:32 AM, David Boxenhorn wrote:

> Just in case you don't know: You can do range searches on keys even =
with Random Partitioner, you just won't get the results in order. If =
this is good enough for you (e.g. if you can order the results on the =
client, or if you just need to get the right answer, but not the right =
order), then you should use Random Partitioner.=20
>=20
> (I bring this up because it confused me until recently.)=20
>=20
> On Wed, May 26, 2010 at 5:14 AM, Steve Lihn <stevelihn@gmail.com> =
wrote:
> I have a question on using Order Preserving Partitioner.=20
>=20
> Many rowKeys in my system will be related to dates, so it seems =
natural to use Order Preserving Partitioner instead of the default =
Random Partitioner. However, I have been warned that special attention =
has to be applied for Order Preserving Partitioner to work properly =
(basically to ensure a good key distribution and avoid "hot spot") and =
reverting it back to Random may not be easy. Also not every rowKey is =
related to dates, for these, using Random Partitioner is okay, but there =
is only one place to set Partitioner.
>=20
> (Note: The intension of this warning is actually to discredit =
Cassandra and persuade me not to use it.)
>=20
> It seems the choice of Partitioner is defined in the storage-conf.xml =
and is a global property. My question why does it have to be a global =
property? Is there a future plan to make it customizable per KeySpace =
(just like you would choose hash or range partition for different =
table/data in RDBMS) ? =20
>=20
> Thanks,
> Steve=20
>=20


--Apple-Mail-83-236363569
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Correct me if I'm wrong here. &nbsp;Even though you can get your =
results with Random Partitioner, it's a lot less efficient if you're =
going across different machines to get your results. &nbsp;If you're =
doing a lot of range queries, it makes sense to have things ordered =
sequentially so that if you do need to go to disk, the reads will be =
faster, rather than lots of random reads across your =
system.<div><br></div><div>It's also my understanding that if you go =
with the OPP, you could hash your key yourself using md5 or sha-1 to =
effectively get random partitioning. &nbsp;So it's a bit of a pain, but =
not impossible to do a split between OPP and RP for your different =
columnfamily/keyspaces.</div><div><br></div><div>On May 26, 2010, at =
2:32 AM, David Boxenhorn wrote:</div><div><div><div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
dir=3D"ltr">Just in case you don't know: You can do range searches on =
keys even with Random Partitioner, you just won't get the results in =
order. If this is good enough for you (e.g. if you can order the results =
on the client, or if you just need to get the right answer, but not the =
right order), then you should use Random Partitioner. <br>
<br>(I bring this up because it confused me until recently.) =
<br><br><div class=3D"gmail_quote">On Wed, May 26, 2010 at 5:14 AM, =
Steve Lihn <span dir=3D"ltr">&lt;<a =
href=3D"mailto:stevelihn@gmail.com">stevelihn@gmail.com</a>&gt;</span> =
wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; =
border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">I have a =
question on using Order Preserving Partitioner. <br><br>Many rowKeys in =
my system will be related to dates, so it seems natural to use Order =
Preserving Partitioner instead of the default Random Partitioner. =
However, I have been warned that special attention has to be applied for =
Order Preserving Partitioner to work properly (basically to ensure a =
good key distribution and avoid "hot spot") and reverting it back to =
Random may not be easy. Also not every rowKey is related to dates, for =
these, using Random Partitioner is okay, but there is only one place to =
set Partitioner.<br>

<br>(Note: The intension of this warning is actually to discredit =
Cassandra and persuade me not to use it.)<br><br>It seems the choice of =
Partitioner is defined in the storage-conf.xml and is a global property. =
My question why does it have to be a global property? Is there a future =
plan to make it customizable per KeySpace (just like you would choose =
hash or range partition for different table/data in RDBMS) ?&nbsp; <br>

<br>Thanks,<br><font color=3D"#888888">Steve <br>
</font></blockquote></div><br></div>
</blockquote></div><br></div></div></body></html>=

--Apple-Mail-83-236363569--