Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=subject
	:references:from:content-type:in-reply-to:message-id:date:to
	:content-transfer-encoding:mime-version; q=dns; s=
	thelastpickle.com; b=LMOarM4Is4nnwCfhsUe0CDz50pw/5wpm9OxFOFw2kp8
	ezw8rfIziQy7/Mpbt/g1vCOY1Ksl/PDQ8fFVo95gWdudo5a9KE0FCTfbOMWNFy4k
	E3chrf9J8uP+lKssJag7zE7L8xL+Jog1AQQTHOfsTpESlXdJVnzXhtIPeLTxhhrA
	=
Subject: Re: Distribution Factor: part of the solution to many-CF problem?
References: <AANLkTikqCWc3o0M+H9Zv=EwNqOO8a=obEvtGga3m=NUw@mail.gmail.com>
 <F965C24E-79FB-4EB2-A274-FA8D7AE12EEE@thelastpickle.com>
 <AANLkTikvNFBSmsbm9DQjixrCVx8MM4QrGwokEvfmv1f2@mail.gmail.com>
 <AANLkTi=_LYMP0Hz2i4Om0FRaevzGkuvmt5PAvrsew9g2@mail.gmail.com>
From: Aaron Morton <aaron@thelastpickle.com>
Content-Type: text/plain;
	charset=us-ascii
In-Reply-To: <AANLkTi=_LYMP0Hz2i4Om0FRaevzGkuvmt5PAvrsew9g2@mail.gmail.com>
Message-Id: <AD2435A4-F338-4547-9A11-A58F154717AC@thelastpickle.com>
Date: Wed, 23 Feb 2011 08:49:55 +1300
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (iPad Mail 8C148)

> The single partitioner is "baked in"
That was my point.

You could perhaps write a partitioner that considers the CF when deciding wh=
at nodes to put data on. Off the top of my head the partitioner is not told a=
bout the  CF the key is storing in.=20

Aaron

On 23/02/2011, at 6:01 AM, Edward Capriolo <edlinuxguru@gmail.com> wrote:

> On Mon, Feb 21, 2011 at 5:14 PM, David Boxenhorn <david@lookin2.com> wrote=
:
>> No, that's not what I mean at all.
>>=20
>> That message is about the ability to use different partitioners for
>> different CFs, say, RandomPartitioner for one, OPP for another.
>>=20
>> I'm talking about defining how many nodes a CF should be distributed over=
,
>> which would be useful if you have a lot of nodes and a lot of small CFs
>> (small relative to the total amount of data).
>>=20
>>=20
>> On Mon, Feb 21, 2011 at 9:58 PM, Aaron Morton <aaron@thelastpickle.com>
>> wrote:
>>>=20
>>> Sounds a bit like this idea
>>> http://www.mail-archive.com/dev@cassandra.apache.org/msg01799.html
>>>=20
>>> Aaron
>>>=20
>>> On 22/02/2011, at 1:28 AM, David Boxenhorn <david@lookin2.com> wrote:
>>>=20
>>>> Cassandra is both distributed and replicated. We have Replication Facto=
r
>>>> but no Distribution Factor!
>>>>=20
>>>> Distribution Factor would define over how many nodes a CF should be
>>>> distributed.
>>>>=20
>>>> Say you want to support millions of multi-tenant users in clusters with=

>>>> thousands of nodes, where you don't know the user's schema in advance, s=
o
>>>> you can't have users share CFs.
>>>>=20
>>>> In this case you wouldn't want to spread out each user's Column Familie=
s
>>>> over thousands of nodes! You would want something like: RF=3D3, DF=3D10=
 i.e.
>>>> distribute each CF over 10 nodes, within those nodes replicate 3 times.=

>>>>=20
>>>> One implementation of DF would be to hash the CF name, and use the same=

>>>> strategies defined for RF to choose the N nodes in DF=3DN.
>>>>=20
>>=20
>>=20
>=20
> The single partitioner is "baked in"
>=20
> Here is a possible solution. Use OOP, but md5 hash your keys client side.
>=20
> This solves that, but when you have keyspaces using OOP but with
> different key distributions this falls apart.