Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D9F24E8B0 for ; Wed, 21 Nov 2012 07:41:09 +0000 (UTC) Received: (qmail 87739 invoked by uid 500); 21 Nov 2012 07:41:07 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 87721 invoked by uid 500); 21 Nov 2012 07:41:07 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 87658 invoked by uid 99); 21 Nov 2012 07:41:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Nov 2012 07:41:04 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of timm.turn@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-ob0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Nov 2012 07:40:58 +0000 Received: by mail-ob0-f172.google.com with SMTP id ef5so7338566obb.31 for ; Tue, 20 Nov 2012 23:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=dW6nr1xEwesX7LmVN1QV24TB0ewZ4Ej+ybY2gHC/cII=; b=uWZifvtNzfiridkpn8ttFmTGujBAbG/0DYl/5SuK0Nrq6pVvX81vGqlILaYygmJwfG hxbrKbY2ZuNNsdvdyqIGDm/LekzXU8nl0b3ajHCOkUCjYf42M6rkP4Axgj0xSjDcNayq SVP9/UW3lIm36lxBW+O/WNNDPEYTE8OPh/RL1qfKDLjhp3XnW6GWFifCXh+bYl6ylNmt 9goCSlrQlDLe3xAfpZVRl9RM/WiboAMBc5zA/NMEG7UFQhc8bkBuF1h1YuN9CjNa5Y0A rgaDHwxJ9pAQEp6FsvwMNfwxILfTQZA2eLVCtKcWJ5eQAG+Bgp9qHplQE7ZVtK+WtsXN TLCA== MIME-Version: 1.0 Received: by 10.60.169.243 with SMTP id ah19mr15465256oec.127.1353483637261; Tue, 20 Nov 2012 23:40:37 -0800 (PST) Received: by 10.60.171.40 with HTTP; Tue, 20 Nov 2012 23:40:37 -0800 (PST) In-Reply-To: References: <501A02AB.7080200@rightscale.com> Date: Wed, 21 Nov 2012 08:40:37 +0100 Message-ID: Subject: Re: Looking for a good Ruby client From: Timmy Turner To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=bcaec517a6b468869604cefc7762 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec517a6b468869604cefc7762 Content-Type: text/plain; charset=ISO-8859-1 Thanks Mat! I thought you were going to expose the internals of CQL3 features like (wide rows with) complex keys and collections to CQL2 clients (which is something that should generally be possible, if Datastax' blog posts are accurate, i.e. an actual description of how things were implemented and not just a conceptual one). I'm still negotiating with my project lead on what features will ultimately be implemented, so I'm not sure whether CQL2/3 interoperability will actually make it into the final 'product' .. but it isn't very high up on the priority list, so it will most likely be implemented towards the end, and thus I guess it'll also kind of depend on how much CQL2 support will be provided by Cassandra itself when the time comes. 2012/11/20 Mat Brown > Hi Timmy, > > I haven't done a lot of playing with CQL3 yet, mostly just reading the > blog posts, so the following is subject to change : ) > > Right now, the Cequel model layer has a skinny row model (which is > designed to follow common patterns of Ruby ORMs) and a wide row model > (which is designed to behave more or less like a Hash, the equivalent > of Java's HashMap). The two don't integrate with each other in any > meaningful way, but as far as I understand it, they do pretty much > cover the data modeling possibilities in CQL2. > > The big idea I've got for the overhaul of Cequel for CQL3 is to allow > building a rich, nested data model by integrating different flavors of > CQL3 table, most notably multi-column primary keys, as well as > collections. The core data types I have in mind are: > > 1) Skinny row with simple primary key (e.g. blogs, with blog_id key) > 2) Skinny row with complex primary key (e.g. blog_posts, with > (blog_id, post_id) key) > 3) Wide row with simple primary key (e.g. blog_languages -- kind of a > weak example but i can't think of anything better for a blog : ) > 4) Wide row with complex primary key (e.g. blog_post_tags) > > My goal is to make it easy to model one-one relationships via a shared > primary key, and one-many via a shared prefix of the primary key. So, > for instance, blogs and blog_languages rows would be one-one (both > with a blog_id primary key) and blogs and blog_posts would be one-many > (sharing the blog_id prefix in the primary key). > > From what I've read, it seems fairly clear that the actual CQL used to > interact with #1 will be the same for CQL2 column families and CQL3 > tables, so no explicit backward compatibility would be needed. #2 and > #4 are, of course, CQL3-only, so backward compatibility isn't an issue > there either. What I'm not entirely clear on is #3 -- this is > straightforward in CQL2, and presumably a CQL3 table with compact > storage would behave in the same way. However, my understanding so far > is that a non-compact CQL3 table would treat this structure > differently, in that both the "key" and "value" of the map would > correspond to columns in a CQL3 table. It may make more sense to just > target compact storage tables with this data structure, but I'm going > to need to play around with it more to figure that out. Otherwise, > Cequel will need to provide two flavors of that structure. > > There's also some tension between CQL3 collections and just using > traditional wide-row structures to achieve the same thing. For > instance, blog_tags could also just be a tags collection in the blogs > table. My plan at this point is to offer both options, since each has > its advantages (collections don't require the creation of a separate > table; but a separate table gives you access to slices of the > collection). > > Anyway, that's probably a lot more of an answer than you needed, but > hopefully the context helps. Definitely interested to hear about the > direction you take your client in as well. > > Finally, regarding a blog, we've got one set up, but it's not live > yet. I'll ping you with a link when it is; I'll certainly be posting > on the development of the next Cequel release. > > Cheers, > Mat > > On Tue, Nov 20, 2012 at 9:23 AM, Timmy Turner wrote: > > @Mat Brown: > > > >> (while still retaining compatibility with CQL2 structures). > > > > Do you mean by exceeding what Cassandra itself provides in terms of > CQL2/3 > > interoperability? > > > > I'm looking into something similar currently (however in Java not in > Ruby) > > and would be interested in your experiences, if you follow through with > the > > plan. Do you have a blog? > > > > > > Thanks! > > > > > > 2012/11/20 Alain RODRIGUEZ > >> > >> @Mat > >> > >> Well I guess you could add your Ruby client to this list since there is > >> not a lot of them yet. > >> > >> http://wiki.apache.org/cassandra/ClientOptions > >> > >> Alain > >> > >> > >> 2012/11/20 Mat Brown > >>> > >>> As the author of Cequel, I can assure you it is excellent ; ) > >>> > >>> We use it in production at Brewster and it is quite stable. If you try > >>> it out and find any bugs, we'll fix 'em quickly. > >>> > >>> I'm planning a big overhaul of the model layer over the holidays to > >>> expose all the > >>> new data modeling goodness in CQL3 (while still retaining > >>> compatibility with CQL2 structures). > >>> > >>> On Thu, Nov 15, 2012 at 3:42 PM, Harry Wilkinson > > >>> wrote: > >>> > Update on this: someone just pointed me towards the Cequel gem: > >>> > https://github.com/brewster/cequel > >>> > > >>> > The way it's described in the readme it looks like exactly what I was > >>> > looking for - a modern, CQL-based gem that is in active development > and > >>> > also > >>> > follows the ActiveModel pattern. I'd be very interested to hear if > >>> > anybody > >>> > has used this, whether it's stable/reliable, etc. > >>> > > >>> > Thanks. > >>> > > >>> > Harry > >>> > > >>> > On 2 August 2012 00:31, Thorsten von Eicken > wrote: > >>> >> > >>> >> Harry, we're in a similar situation and are starting to work out our > >>> >> own > >>> >> ruby client. The biggest issue is that it doesn't make much sense to > >>> >> build a > >>> >> higher level abstraction on anything other than CQL3, given where > >>> >> things are > >>> >> headed. At least this is our opinion. > >>> >> At the same time, CQL3 is just barely becoming usable and still > seems > >>> >> rather deficient in wide-row usage. The tricky part is that with the > >>> >> current > >>> >> CQL3 you have to construct quite complex iterators to retrieve a > large > >>> >> result set. Which means that you end up having to either parse CQL3 > >>> >> coming > >>> >> in to insert the iteration stuff, or you have to pass CQL3 fragments > >>> >> in and > >>> >> compose them together with iterator clauses. Not fun stuff either > way. > >>> >> The only good solution I see is to switch to a streaming protocol > (or > >>> >> build some form of "continue" on top of thrift) such that the client > >>> >> can ask > >>> >> for a huge result set and the cassandra coordinator can break it > into > >>> >> sub-queries as it sees fit and return results chunk-by-chunk. If > this > >>> >> is > >>> >> really the path forward then all abstractions built above CQL3 > before > >>> >> that > >>> >> will either have a good piece of complex code that can be deleted or > >>> >> worse, > >>> >> will have an interface that is no longer best practice. > >>> >> Good luck! > >>> >> Thorsten > >>> >> > >>> >> > >>> >> > >>> >> On 8/1/2012 1:47 PM, Harry Wilkinson wrote: > >>> >> > >>> >> Hi, > >>> >> > >>> >> I'm looking for a Ruby client for Cassandra that is pretty > high-level. > >>> >> I > >>> >> am really hoping to find a Ruby gem of high quality that allows a > >>> >> developer > >>> >> to create models like you would with ActiveModel. > >>> >> > >>> >> So far I have figured out that the canonical Ruby client for > Cassandra > >>> >> is > >>> >> Twitter's Cassandra gem of the same name. It looks great - mature, > >>> >> still in > >>> >> active development, etc. No stated support for Ruby 1.9.3 that I > can > >>> >> see, > >>> >> but I can probably live with that for now. > >>> >> > >>> >> What I'm looking for is a higher-level gem built on that gem that > >>> >> works > >>> >> like ActiveModel in that you just include a module in your model > class > >>> >> and > >>> >> that gives you methods to declare your model's serialized attributes > >>> >> and > >>> >> also the usual ActiveModel methods like 'save!', 'valid?', 'find', > >>> >> etc. > >>> >> > >>> >> I've been trying out some different NoSQL databases recently, and > for > >>> >> example there is an official Ruby client for Riak with a domain > model > >>> >> that > >>> >> is close to Riak's, but then there's also a gem called 'Ripple' that > >>> >> uses a > >>> >> domain model that is closer to what most Ruby developers are used > to. > >>> >> So it > >>> >> looks like Twitter's Cassandra gem is the one that stays close to > the > >>> >> domain > >>> >> model of Cassandra, and what I'm looking for is a gem that's a > >>> >> Cassandra > >>> >> equivalent of RIpple. > >>> >> > >>> >> From some searching I found cassandra_object, which has been > inactive > >>> >> for > >>> >> a couple of years, but there's a fork that looks like it's being > >>> >> maintained, > >>> >> but I have not found any kind of information to suggest the > maintained > >>> >> fork > >>> >> is in general use yet. I have found quite a lot of gems of a > similar > >>> >> style > >>> >> that people have started and then not really got very far with. > >>> >> > >>> >> So, does anybody know of a suitable gem? Would you recommend it? > Or > >>> >> perhaps you would recommend not using such a gem and sticking with > the > >>> >> lower-level client gem? > >>> >> > >>> >> Thanks in advance for your advice. > >>> >> > >>> >> Harry > >>> >> > >>> >> > >>> > > >> > >> > > > --bcaec517a6b468869604cefc7762 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks Mat!

I thought you were going to expose the inter= nals of CQL3 features like (wide rows with) complex keys and collections to= CQL2 clients (which is something that should generally be possible, if Dat= astax' blog posts are accurate, i.e. an actual description of how thing= s were implemented and not just a conceptual one).

I'm still negotiating with my project lead on what = features will ultimately be implemented, so I'm not sure whether CQL2/3= interoperability will actually make it into the final 'product' ..= but it isn't very high up on the priority list, so it will most likely= be implemented towards the end, and thus I guess it'll also kind of de= pend on how much CQL2 support will be provided by Cassandra itself when the= time comes.


2012/11/20 Ma= t Brown <mat@brewster.com>
Hi Timmy,

I haven't done a lot of playing with CQL3 yet, mostly just reading the<= br> blog posts, so the following is subject to change : )

Right now, the Cequel model layer has a skinny row model (which is
designed to follow common patterns of Ruby ORMs) and a wide row model
(which is designed to behave more or less like a Hash, the equivalent
of Java's HashMap). The two don't integrate with each other in any<= br> meaningful way, but as far as I understand it, they do pretty much
cover the data modeling possibilities in CQL2.

The big idea I've got for the overhaul of Cequel for CQL3 is to allow building a rich, nested data model by integrating different flavors of
CQL3 table, most notably multi-column primary keys, as well as
collections. The core data types I have in mind are:

1) Skinny row with simple primary key (e.g. blogs, with blog_id key)
2) Skinny row with complex primary key (e.g. blog_posts, with
(blog_id, post_id) key)
3) Wide row with simple primary key (e.g. blog_languages -- kind of a
weak example but i can't think of anything better for a blog : )
4) Wide row with complex primary key (e.g. blog_post_tags)

My goal is to make it easy to model one-one relationships via a shared
primary key, and one-many via a shared prefix of the primary key. So,
for instance, blogs and blog_languages rows would be one-one (both
with a blog_id primary key) and blogs and blog_posts would be one-many
(sharing the blog_id prefix in the primary key).

>From what I've read, it seems fairly clear that the actual CQL used to<= br> interact with #1 will be the same for CQL2 column families and CQL3
tables, so no explicit backward compatibility would be needed. #2 and
#4 are, of course, CQL3-only, so backward compatibility isn't an issue<= br> there either. What I'm not entirely clear on is #3 -- this is
straightforward in CQL2, and presumably a CQL3 table with compact
storage would behave in the same way. However, my understanding so far
is that a non-compact CQL3 table would treat this structure
differently, in that both the "key" and "value" of the = map would
correspond to columns in a CQL3 table. It may make more sense to just
target compact storage tables with this data structure, but I'm going to need to play around with it more to figure that out. Otherwise,
Cequel will need to provide two flavors of that structure.

There's also some tension between CQL3 collections and just using
traditional wide-row structures to achieve the same thing. For
instance, blog_tags could also just be a tags collection in the blogs
table. My plan at this point is to offer both options, since each has
its advantages (collections don't require the creation of a separate table; but a separate table gives you access to slices of the
collection).

Anyway, that's probably a lot more of an answer than you needed, but hopefully the context helps. Definitely interested to hear about the
direction you take your client in as well.

Finally, regarding a blog, we've got one set up, but it's not live<= br> yet. I'll ping you with a link when it is; I'll certainly be postin= g
on the development of the next Cequel release.

Cheers,
Mat

On Tue, Nov 20, 2012 at 9:23 AM, Timmy Turner <timm.turn@gmail.com> wrote:
> @Mat Brown:
>
>> (while still retaining compatibility with CQL2 structures).
>
> Do you mean by exceeding what Cassandra itself provides in terms of CQ= L2/3
> interoperability?
>
> I'm looking into something similar currently (however in Java not = in Ruby)
> and would be interested in your experiences, if you follow through wit= h the
> plan. Do you have a blog?
>
>
> Thanks!
>
>
> 2012/11/20 Alain RODRIGUEZ <a= rodrime@gmail.com>
>>
>> @Mat
>>
>> Well I guess you could add your Ruby client to this list since the= re is
>> not a lot of them yet.
>>
>> http://wiki.apache.org/cassandra/ClientOptions
>>
>> Alain
>>
>>
>> 2012/11/20 Mat Brown <mat@b= rewster.com>
>>>
>>> As the author of Cequel, I can assure you it is excellent ; )<= br> >>>
>>> We use it in production at Brewster and it is quite stable. If= you try
>>> it out and find any bugs, we'll fix 'em =A0quickly. >>>
>>> I'm planning a big overhaul of the model layer over the ho= lidays to
>>> expose all the
>>> new data modeling goodness in CQL3 (while still retaining
>>> compatibility with CQL2 structures).
>>>
>>> On Thu, Nov 15, 2012 at 3:42 PM, Harry Wilkinson <hwilkinson@mdsol.com>
>>> wrote:
>>> > Update on this: someone just pointed me towards the Ceque= l gem:
>>> > https://github.com/brewster/cequel
>>> >
>>> > The way it's described in the readme it looks like ex= actly what I was
>>> > looking for - a modern, CQL-based gem that is in active d= evelopment and
>>> > also
>>> > follows the ActiveModel pattern. =A0I'd be very inter= ested to hear if
>>> > anybody
>>> > has used this, whether it's stable/reliable, etc.
>>> >
>>> > Thanks.
>>> >
>>> > Harry
>>> >
>>> > On 2 August 2012 00:31, Thorsten von Eicken <tve@rightscale.com> wrote:
>>> >>
>>> >> Harry, we're in a similar situation and are start= ing to work out our
>>> >> own
>>> >> ruby client. The biggest issue is that it doesn't= make much sense to
>>> >> build a
>>> >> higher level abstraction on anything other than CQL3,= given where
>>> >> things are
>>> >> headed. At least this is our opinion.
>>> >> At the same time, CQL3 is just barely becoming usable= and still seems
>>> >> rather deficient in wide-row usage. The tricky part i= s that with the
>>> >> current
>>> >> CQL3 you have to construct quite complex iterators to= retrieve a large
>>> >> result set. Which means that you end up having to eit= her parse CQL3
>>> >> coming
>>> >> in to insert the iteration stuff, or you have to pass= CQL3 fragments
>>> >> in and
>>> >> compose them together with iterator clauses. Not fun = stuff either way.
>>> >> The only good solution I see is to switch to a stream= ing protocol (or
>>> >> build some form of "continue" on top of thr= ift) such that the client
>>> >> can ask
>>> >> for a huge result set and the cassandra coordinator c= an break it into
>>> >> sub-queries as it sees fit and return results chunk-b= y-chunk. If this
>>> >> is
>>> >> really the path forward then all abstractions built a= bove CQL3 before
>>> >> that
>>> >> will either have a good piece of complex code that ca= n be deleted or
>>> >> worse,
>>> >> will have an interface that is no longer best practic= e.
>>> >> Good luck!
>>> >> Thorsten
>>> >>
>>> >>
>>> >>
>>> >> On 8/1/2012 1:47 PM, Harry Wilkinson wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> I'm looking for a Ruby client for Cassandra that = is pretty high-level.
>>> >> I
>>> >> am really hoping to find a Ruby gem of high quality t= hat allows a
>>> >> developer
>>> >> to create models like you would with ActiveModel.
>>> >>
>>> >> So far I have figured out that the canonical Ruby cli= ent for Cassandra
>>> >> is
>>> >> Twitter's Cassandra gem of the same name. =A0It l= ooks great - mature,
>>> >> still in
>>> >> active development, etc. =A0No stated support for Rub= y 1.9.3 that I can
>>> >> see,
>>> >> but I can probably live with that for now.
>>> >>
>>> >> What I'm looking for is a higher-level gem built = on that gem that
>>> >> works
>>> >> like ActiveModel in that you just include a module in= your model class
>>> >> and
>>> >> that gives you methods to declare your model's se= rialized attributes
>>> >> and
>>> >> also the usual ActiveModel methods like 'save!= 9;, 'valid?', 'find',
>>> >> etc.
>>> >>
>>> >> I've been trying out some different NoSQL databas= es recently, and for
>>> >> example there is an official Ruby client for Riak wit= h a domain model
>>> >> that
>>> >> is close to Riak's, but then there's also a g= em called 'Ripple' that
>>> >> uses a
>>> >> domain model that is closer to what most Ruby develop= ers are used to.
>>> >> So it
>>> >> looks like Twitter's Cassandra gem is the one tha= t stays close to the
>>> >> domain
>>> >> model of Cassandra, and what I'm looking for is a= gem that's a
>>> >> Cassandra
>>> >> equivalent of RIpple.
>>> >>
>>> >> From some searching I found cassandra_object, which h= as been inactive
>>> >> for
>>> >> a couple of years, but there's a fork that looks = like it's being
>>> >> maintained,
>>> >> but I have not found any kind of information to sugge= st the maintained
>>> >> fork
>>> >> is in general use yet. =A0I have found quite a lot of= gems of a similar
>>> >> style
>>> >> that people have started and then not really got very= far with.
>>> >>
>>> >> So, does anybody know of a suitable gem? =A0Would you= recommend it? =A0Or
>>> >> perhaps you would recommend not using such a gem and = sticking with the
>>> >> lower-level client gem?
>>> >>
>>> >> Thanks in advance for your advice.
>>> >>
>>> >> Harry
>>> >>
>>> >>
>>> >
>>
>>
>

--bcaec517a6b468869604cefc7762--