Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of timm.turn@gmail.com
 designates 209.85.214.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1D1AC0D6-B39B-4946-8800-462111BE38CA@thelastpickle.com>
References: 
 <CA+uKMZAefCgfLqaR_tKuVyeV=eRopnbTc_Cfc-2N2E=sY69YuQ@mail.gmail.com>
	<CAKkz8Q2kXu2f8YzQ4Aqe0_AgcjoPZX_eJAQ=Yp61AKTQeG-36g@mail.gmail.com>
	<CA+uKMZDTw4DPc2mg3D+bo28zFqWHOc1BH4NcUeDhoJepCgErsQ@mail.gmail.com>
	<9628F5E8-54D6-444E-A345-CCF906704B4F@thelastpickle.com>
	<CA+uKMZC-u+gNHhQnYz-V7ANyVpy+ybPDgy5dA26-XjoDf_1BDg@mail.gmail.com>
	<1D1AC0D6-B39B-4946-8800-462111BE38CA@thelastpickle.com>
Date: Wed, 31 Oct 2012 18:23:51 +0100
Message-ID: 
 <CA+uKMZBqhP0Oyy0ypfsJ0D_tREf80ZaVq7CjTEwOORTc+f+3xw@mail.gmail.com>
Subject: Re: CQL3: Unknown property 'comparator'?
From: Timmy Turner <timm.turn@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1

> CQL 3 is not locked, the more feedback the better.

I'm pretty sure that my use case is not relevant enough to consider
special cases for it in Cassandra/CQL. In fact it's quite the other
way around - it's my job to determine how well my tool fits Cassandra
an other databases.

> Can you provide details on the use case?

Basically what I tried to do was some sort of transparent client-side
encryption, for those few that want to run their database in the
cloud, but are too paranoid to do so - i.e. hook the Cassandra (CQL)
library, parse the query myself (using a parser generated by
Cassandra's own grammar definition), extract the values the user said
he wanted encrypted (he did so in a configuration file), encrypt them,
replace the original values with the encrypted ones in the query
string, and hand it over to the actual CQL client library. If a result
set came back, then I decrypted it before ultimately handing it over
to the user.

I also had the option to encrypt column names with individual keys per
tuple, so they wouldn't result in the same ciphertext across tuples
(for those ultra-paranoid users). Other than severely limiting the
functionality of Cassandra (i.e. a user couldn't select specific
columns any more, if more than one tuple was to be queried from the
database; but queries for one specific tuple would still work without
any kind of limitation), this worked fine in CQL2. In CQL3 this,
obviously, isn't possible any more, if the users won't settle for
using composite keys instead of columns.

> AFAIK to be  "schema free"  you use the "row-column-value" model. While this
> looks like many rows it is a single store engine row.

I know. CQL3 is still a superset of CQL2's features, so in theory it
can still do everything CQL2 could - but the real question is how
Cassandra users will end up using it, and what kind of usage patterns
the current data model encourages.

2012/10/30 aaron morton <aaron@thelastpickle.com>:
> My use case relied on "mangling" the column names for various
>
> Can you provide details on the use case?
> CQL 3 is not locked, the more feedback the better.
>
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
>
> AFAIK to be  "schema free"  you use the "row-column-value" model. While this
> looks like many rows it is a single store engine row.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/10/2012, at 8:45 PM, Timmy Turner <timm.turn@gmail.com> wrote:
>
> Is your use case covered in the article above ?
>
>
> My use case relied on "mangling" the column names for various
> purposes, and CQL3 with its transposed columns does of course still
> allow for that, but it means rewriting the part of the application
> that dealt with CQL so it can handle the new syntax/data model.
>
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
>
>
> Thanks!
>
> 2012/10/30 aaron morton <aaron@thelastpickle.com>:
>
> More background http://www.datastax.com/dev/blog/thrift-to-cql3
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory?
>
> Not really, it sort of depends on your view.
>
> Lets say this is a "schema free" CF definition in CLI
>
>  create column family clicks
>    with key_validation_class = UTF8Type
>     and comparator = DateType
>     and default_validation_class = UTF8Type
>
> It could be used for wide rows with lots of columns, where the name is a
> date.
>
> As the article at the top says, this CQL 3 DDL is equivalent:
>
>    CREATE TABLE clicks (
>      key text,
>      column1 timestamp,
>      value text,
>      PRIMARY KEY (key, column)
>    ) WITH COMPACT STORAGE
>
> This creates a single row inside C*, column name is a date. The difference
> is CQL 3 pivots this one "storage engine" row into multiple CQL 3 rows. (See
> article)
>
> So far so good. Let's add some schema:
>
>    CREATE TABLE clicks (
>      user_id text,
>      click_time timestamp,
>      click_url text,
>      PRIMARY KEY (user_id, click_time)
>    ) WITH COMPACT STORAGE
>
> That's functionally the same but has some more schema in it. It tells CQL 3
> that the label to use for the name of a column is "click_time". Previously
> the label was "column1".
>
>
> It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
> Is your use case covered in the article above ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/10/2012, at 2:31 AM, Timmy Turner <timm.turn@gmail.com> wrote:
>
> Thank you! That article helps clear up a lot of my confusion about the
> changes between CQL 2 and 3, since I was wondering how to
> access/manipulate CompositeType/DynamicCompositeType columns through
> CQL.
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory? It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
>
>
>
> 2012/10/29 Sylvain Lebresne <sylvain@datastax.com>:
>
> CQL3 does absolutely allow dynamic column families, but does it
> differently from CQL2. See
> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>
> --
> Sylvain
>
> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <timm.turn@gmail.com> wrote:
>
> Does CQL3 not allow dynamic columns (column names) any more?
>
>
>