cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DuyHai Doan <doanduy...@gmail.com>
Subject Re: Dynamic Columns in Cassandra 2.X
Date Fri, 13 Jun 2014 20:19:31 GMT
Hi Mark

 I believe that in your table you want to have some "common" fields that
will be there whatever customer is, and other fields that are entirely
customer-dependent, isn't it ?

 In this case, creating a table with static columns for the common fields
and a clustering column representing all custom fields defined by a
customer could be a solution (see here for static column:
https://issues.apache.org/jira/browse/CASSANDRA-6561 )

CREATE TABLE user_data (
   user_id bigint,
   user_firstname text static,
   user_lastname text static,
   ...
   custom_property_name text,
   custom_property_value text,
   PRIMARY KEY(user_id, custom_property_name, custom_property_value));

 Please note that with this solution you need to have "at least one" custom
property per customer to make it work

 The only thing to take care of is the type of custom_property_value. You
need to define it once for all. To accommodate for dynamic types, you can
either save the value as blob or text(as JSON) and take care of the
serialization/deserialization yourself at the client side

 As an alternative you can save custom properties in a map, provided that
their number is not too large. But considering the business case of CRM, I
believe that it's quite rare and user has more than 1000 custom properties
isn't it ?



On Fri, Jun 13, 2014 at 10:03 PM, Mark Greene <greenemj@gmail.com> wrote:

> My use case requires the support of arbitrary columns much like a CRM. My
> users can define 'custom' fields within the application. Ideally I wouldn't
> have to change the schema at all, which is why I like the old thrift
> approach rather than the CQL approach.
>
> Having said all that, I'd be willing to adapt my API to make explicit
> schema changes to Cassandra whenever my user makes a change to their custom
> fields if that's an accepted practice.
>
> Ultimately, I'm trying to figure out of the Cassandra community intends to
> support true schemaless use cases in the future.
>
> --
> about.me <http://about.me/markgreene>
>
>
> On Fri, Jun 13, 2014 at 3:47 PM, DuyHai Doan <doanduyhai@gmail.com> wrote:
>
>> This strikes me as bad practice in the world of multi tenant systems. I
>> don't want to create a table per customer. So I'm wondering if dynamically
>> modifying the table is an accepted practice?  --> Can you give some details
>> about your use case ? How would you "alter" a table structure to adapt it
>> to a new customer ?
>>
>> Wouldn't it be better to model your table so that it supports
>> addition/removal of customer ?
>>
>>
>>
>> On Fri, Jun 13, 2014 at 9:00 PM, Mark Greene <greenemj@gmail.com> wrote:
>>
>>> Thanks DuyHai,
>>>
>>> I have a follow up question to #2. You mentioned ideally I would create
>>> a new table instead of mutating an existing one.
>>>
>>> This strikes me as bad practice in the world of multi tenant systems. I
>>> don't want to create a table per customer. So I'm wondering if dynamically
>>> modifying the table is an accepted practice?
>>>
>>> --
>>> about.me <http://about.me/markgreene>
>>>
>>>
>>> On Fri, Jun 13, 2014 at 2:54 PM, DuyHai Doan <doanduyhai@gmail.com>
>>> wrote:
>>>
>>>> Hello Mark
>>>>
>>>>  Dynamic columns, as you said, are perfectly supported by CQL3 via
>>>> clustering columns. And no, using collections for storing dynamic data is
a
>>>> very bad idea if the cardinality is very high (>> 1000 elements)
>>>>
>>>> 1)  Is using Thrift a valid approach in the era of CQL?  --> Less and
>>>> less. Unless you are looking for extreme performance, you'd better off
>>>> choosing CQL3. The ease of programming and querying with CQL3 does worth
>>>> the small overhead in CPU
>>>>
>>>> 2) If CQL is the best practice,  should I alter the schema at runtime
>>>> when I detect I need to do an schema mutation?  --> Ideally you should
not
>>>> alter schema but create a new table to adapt to your changing requirements.
>>>>
>>>> 3) If I utilize CQL collections, will Cassandra page the entire thing
>>>> into the heap?  --> Of course. All collections and maps in Cassandra are
>>>> eagerly loaded entirely in memory on server side. That's why it is
>>>> recommended to limit their cardinality to ~ 1000 elements
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jun 13, 2014 at 8:33 PM, Mark Greene <greenemj@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm looking for some best practices w/r/t supporting arbitrary
>>>>> columns. It seems from the docs I've read around CQL that they are
>>>>> supported in some capacity via collections but you can't exceed 64K in
>>>>> size. For my requirements that would cause problems.
>>>>>
>>>>> So my questions are:
>>>>>
>>>>> 1)  Is using Thrift a valid approach in the era of CQL?
>>>>>
>>>>> 2) If CQL is the best practice,  should I alter the schema at runtime
>>>>> when I detect I need to do an schema mutation?
>>>>>
>>>>>  3) If I utilize CQL collections, will Cassandra page the entire
>>>>> thing into the heap?
>>>>>
>>>>> My data model is akin to a CRM, arbitrary column definitions per
>>>>> customer.
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Mark
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message