Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
MIME-Version: 1.0
In-Reply-To: 
 <CAB-G1ji+ubjRbe2G1xKrDvgf6thtsvcNGpsJAGnLky5TbqMzUA@mail.gmail.com>
References: 
 <CAB-G1ji+ubjRbe2G1xKrDvgf6thtsvcNGpsJAGnLky5TbqMzUA@mail.gmail.com>
Date: Wed, 13 Nov 2013 08:28:27 -0600
Message-ID: 
 <CAKmMYa9rtfjMD3FJf_Hs+Q8kHVOyRq6Fi2O9ODWgi+QqTYEf7g@mail.gmail.com>
Subject: Re: Modeling multi-tenanted Cassandra schema
From: Nate McCall <nate@thelastpickle.com>
To: Cassandra Users <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=001a11c29a164e97ad04eb0fc7db

--001a11c29a164e97ad04eb0fc7db
Content-Type: text/plain; charset=ISO-8859-1

You basically want option (c). Option (d) might work, but you would be
bending the paradigm a bit, IMO. Certainly do not use dedicated column
families or keyspaces per tennant. That never works. The list history will
show that with a few google searches and we've seen it fail badly with
several clients.

Overall, option (c) would be difficult to do in CQL without some very well
thought out abstractions and/or a deep hack on the Java driver (not
in-ellegant or impossible, just lots of moving parts to get your head
around if you are new to such). That said, depending on the size of your
project and skill of your team, this direction might be worth considering.

Usergrid (just accepted for incubation at Apache) functions this way via
the Thrift API: https://github.com/apigee/usergrid-stack

The commercial version of Usergrid has "tens of thousands" of active
tennants on a single cluster (same code base at the service layer as the
open source version). It uses Hector's built in virtual keyspaces:
https://github.com/hector-client/hector/wiki/Virtual-Keyspaces (NOTE:
though Hector is sunsetting/in patch maintenance, the approach is certainly
legitimate - but I'd recommend you *not* start a new project on Hector).

In short, Usergrid is the only project I know of that has a well-proven
tenant model that functions at scale, though I'm sure there are others
around, just not open sourced or actually running large deployments.

Astyanax can do this as well albeit with a little more work required:
https://github.com/Netflix/astyanax/wiki/Composite-columns#how-to-use-the-prefixedserializer-but-you-really-should-use-composite-columns


Happy to clarify any of the above.


On Tue, Nov 12, 2013 at 3:19 AM, Ben Hood <0x6e6562@gmail.com> wrote:

> Hi,
>
> I've just received a requirement to make a Cassandra app
> multi-tenanted, where we'll have up to 100 tenants.
>
> Most of the tables are timestamped wide row tables with a natural
> application key for the partitioning key and a timestamp key as a
> cluster key.
>
> So I was considering the options:
>
> (a) Add a tenant column to each table and stick a secondary index on
> that column;
> (b) Add a tenant column to each table and maintain index tables that
> use the tenant id as a partitioning key;
> (c) Decompose the partitioning key of each table and add the tenant
> and the leading component of the key;
> (d) Add the tenant as a separate clustering key;
> (e) Replicate the schema in separate tenant specific key spaces;
> (f) Something I may have missed;
>
> Option (a) seems the easiest, but I'm wary of just adding secondary
> indexes without thinking about it.
>
> Option (b) seems to have the least impact of the layout of the
> storage, but a cost of maintaining each index table, both code wise
> and in terms of performance.
>
> Option (c) seems quite straight forward, but I feel it might have a
> significant effect on the distribution of the rows, if the cardinality
> of the tenants is low.
>
> Option (d) seems simple enough, but it would mean that you couldn't
> query for a range of tenants without supplying a range of natural
> application keys, through which you would need to iterate (under the
> assumption that you don't use an ordered partitioner).
>
> Option (e) appears relatively straight forward, but it does mean that
> the application CQL client needs to maintain separate cluster
> connections for each tenant. Also I'm not sure to what extent key
> spaces were designed to partition identically structured data.
>
> Does anybody have any experience with running a multi-tenanted
> Cassandra app, or does this just depend too much on the specifics of
> the application?
>
> Cheers,
>
> Ben
>


-- 
-----------------
Nate McCall
Austin, TX
@zznate

Co-Founder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

--001a11c29a164e97ad04eb0fc7db
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>You basically want option (c). Option (d) might work,=
 but you would be bending the paradigm a bit, IMO. Certainly do not use ded=
icated column families or keyspaces per tennant. That never works. The list=
 history will show that with a few google searches and we&#39;ve seen it fa=
il badly with several clients.</div>
<div>=A0</div>Overall, option (c) would be difficult to do in CQL without s=
ome very well thought out abstractions and/or a deep hack on the Java drive=
r (not in-ellegant or impossible, just lots of moving parts to get your hea=
d around if you are new to such). That said, depending on the size of your =
project and skill of your team, this direction might be worth considering.=
=A0<div>
<br></div><div>Usergrid (just accepted for incubation at Apache) functions =
this way via the Thrift API:=A0<a href=3D"https://github.com/apigee/usergri=
d-stack">https://github.com/apigee/usergrid-stack</a></div><div><br></div><=
div>
The commercial version of Usergrid has &quot;tens of thousands&quot; of act=
ive tennants on a single cluster (same code base at the service layer as th=
e open source version). It uses Hector&#39;s built in virtual keyspaces: <a=
 href=3D"https://github.com/hector-client/hector/wiki/Virtual-Keyspaces">ht=
tps://github.com/hector-client/hector/wiki/Virtual-Keyspaces</a> (NOTE: tho=
ugh Hector is sunsetting/in patch maintenance, the approach is certainly le=
gitimate - but I&#39;d recommend you *not* start a new project on Hector).=
=A0</div>
<div><br></div><div>In short, Usergrid is the only project I know of that h=
as a well-proven tenant model that functions at scale, though I&#39;m sure =
there are others around, just not open sourced or actually running large de=
ployments.=A0</div>
<div><br></div><div>Astyanax can do this as well albeit with a little more =
work required:=A0</div><div><a href=3D"https://github.com/Netflix/astyanax/=
wiki/Composite-columns#how-to-use-the-prefixedserializer-but-you-really-sho=
uld-use-composite-columns">https://github.com/Netflix/astyanax/wiki/Composi=
te-columns#how-to-use-the-prefixedserializer-but-you-really-should-use-comp=
osite-columns</a>=A0</div>
<div><br></div><div>Happy to clarify any of the above.=A0</div></div><div c=
lass=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Tue, Nov 12, 201=
3 at 3:19 AM, Ben Hood <span dir=3D"ltr">&lt;<a href=3D"mailto:0x6e6562@gma=
il.com" target=3D"_blank">0x6e6562@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi,<br>
<br>
I&#39;ve just received a requirement to make a Cassandra app<br>
multi-tenanted, where we&#39;ll have up to 100 tenants.<br>
<br>
Most of the tables are timestamped wide row tables with a natural<br>
application key for the partitioning key and a timestamp key as a<br>
cluster key.<br>
<br>
So I was considering the options:<br>
<br>
(a) Add a tenant column to each table and stick a secondary index on<br>
that column;<br>
(b) Add a tenant column to each table and maintain index tables that<br>
use the tenant id as a partitioning key;<br>
(c) Decompose the partitioning key of each table and add the tenant<br>
and the leading component of the key;<br>
(d) Add the tenant as a separate clustering key;<br>
(e) Replicate the schema in separate tenant specific key spaces;<br>
(f) Something I may have missed;<br>
<br>
Option (a) seems the easiest, but I&#39;m wary of just adding secondary<br>
indexes without thinking about it.<br>
<br>
Option (b) seems to have the least impact of the layout of the<br>
storage, but a cost of maintaining each index table, both code wise<br>
and in terms of performance.<br>
<br>
Option (c) seems quite straight forward, but I feel it might have a<br>
significant effect on the distribution of the rows, if the cardinality<br>
of the tenants is low.<br>
<br>
Option (d) seems simple enough, but it would mean that you couldn&#39;t<br>
query for a range of tenants without supplying a range of natural<br>
application keys, through which you would need to iterate (under the<br>
assumption that you don&#39;t use an ordered partitioner).<br>
<br>
Option (e) appears relatively straight forward, but it does mean that<br>
the application CQL client needs to maintain separate cluster<br>
connections for each tenant. Also I&#39;m not sure to what extent key<br>
spaces were designed to partition identically structured data.<br>
<br>
Does anybody have any experience with running a multi-tenanted<br>
Cassandra app, or does this just depend too much on the specifics of<br>
the application?<br>
<br>
Cheers,<br>
<br>
Ben<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=3D"=
ltr">-----------------<br>Nate McCall<br>Austin, TX<br>@zznate<br><br>Co-Fo=
under &amp; Sr. Technical Consultant<br>Apache Cassandra Consulting<br><a h=
ref=3D"http://www.thelastpickle.com" target=3D"_blank">http://www.thelastpi=
ckle.com</a></div>

</div>

--001a11c29a164e97ad04eb0fc7db--