cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Atul Saroha <atul.sar...@snapdeal.com>
Subject Re: How to migrate MYSQL to cassandra -special case in scoop+datastax
Date Tue, 09 Feb 2016 07:09:42 GMT
Hi Jack,

Table in cassandra actually has 20+ columns which already have Partition
key and cluster key defined.
There are two use cases we need cater but both belongs to separate column
family.

   1. Where key and value are of small in few bytes size.
      - This collection is of data which we fetch all and replace all
      mostly. There could be 3 maps in this table one with 10, 50 and
100 entries
      respectively.
   2. Key is small but value is blob, i.e. JSON of java object.
      - this may contain max 200 entries and avg around 60.








---------------------------------------------------------------------------------------------------------------------
Atul Saroha
*Sr. Software Engineer*
*M*: +91 8447784271 *T*: +91 124-415-6069 *EXT*: 12369
Plot # 362, ASF Centre - Tower A, Udyog Vihar,
 Phase -4, Sector 18, Gurgaon, Haryana 122016, INDIA

On Mon, Feb 8, 2016 at 9:06 PM, Jack Krupansky <jack.krupansky@gmail.com>
wrote:

> Is there a specific reason you are reluctant to simply model that MySQL
> table as-is in Cassandra? It should work fine in Cassandra.
>
> What is the cardinality of your user IDs?
>
> Yes, a collection should be limited to a modest to moderate number of
> elements, like dozens or hundreds at most. If your collection has many
> thousands of elements then it should be modeled using a clustering key, or
> a partition key if its cardinality is in the millions (or even hundreds of
> thousands if the row is bulky.)
>
> -- Jack Krupansky
>
> On Sun, Feb 7, 2016 at 6:18 AM, Atul Saroha <atul.saroha@snapdeal.com>
> wrote:
>
>> Hi Sachin.
>>
>> We can index on collection (either key or value in a map) and can use it
>> in query. But my concern is to migrate the data in this form from a MYSQL
>> table. Using some multi-threaded script and java program is an option but
>> scoop approach would have been better.
>>
>> Moreover, I am even concerned about this statement mentioned in below
>> link:
>>
>> Keep collections small to prevent delays during querying because *Cassandra
>> reads a collection in its entirety. The collection is not paged internally.*
>>
>> https://docs.datastax.com/en/cql/3.0/cql/cql_using/use_collections_c.html
>>
>>
>>
>> Thanks and Regards
>>
>> ---------------------------------------------------------------------------------------------------------------------
>> Atul Saroha
>> *Sr. Software Engineer*
>> *M*: +91 8447784271 *T*: +91 124-415-6069 *EXT*: 12369
>> Plot # 362, ASF Centre - Tower A, Udyog Vihar,
>>  Phase -4, Sector 18, Gurgaon, Haryana 122016, INDIA
>>
>> On Sat, Feb 6, 2016 at 5:42 AM, Sachin Uplaonkar <up.sachi@gmail.com>
>> wrote:
>>
>>> Hello Atul,
>>>
>>> Please follow the document:
>>> http://docs.datastax.com/en/cql/3.3/cql/cql_using/useIndexColl.html
>>>
>>> I tried below code:
>>>
>>> */**CREATE TABLE demo.phone (*
>>> * details MAP<text, text>,*
>>> * initials text,*
>>> * time timeuuid,*
>>> * PRIMARY KEY (initials, time)*
>>> *);**/*
>>>
>>> *//insert into phone (initials, time, details) values ('su', now(),
>>> {'Sachi': '408-797-9600 <408-797-9600>'});*
>>> *//insert into phone(initials, time, details) values ('su', now(),
>>> {'Sam': '555-121-2222'});*
>>> *//insert into phone (initials, time, details) values ('NG', now(),
>>> {'National Geographic': '404-509-2000 <404-509-2000>'});*
>>> *//create index IF NOT EXISTS map_index on phone (details);*
>>>
>>> *select * from phone where details contains key 'Sam';*
>>>
>>>
>>> *[image: Inline image 1]*
>>>
>>>
>>>
>>> On Fri, Feb 5, 2016 at 9:47 AM, Sachin Uplaonkar <up.sachi@gmail.com>
>>> wrote:
>>>
>>>> Adding to the above:
>>>>
>>>> - Search on Collection types is not yet supported in CQL. [Hence, if we
>>>> create the table as per your model, we will not be able to query on it]
>>>>
>>>> Refer:* http://www.datastax.com/dev/blog/cql3_collections
>>>> <http://www.datastax.com/dev/blog/cql3_collections> *[Section: *Things
>>>> to Know*]
>>>>
>>>> On Fri, Feb 5, 2016 at 8:25 AM, Sachin Uplaonkar <up.sachi@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Atul,
>>>>>
>>>>> I am new to Cassandra but below is the thing which I tried:
>>>>>
>>>>> ** Not sure if this was intended by you **
>>>>>
>>>>> CREATE TABLE demo.phone (
>>>>> details MAP<text, text>,
>>>>> initials text,
>>>>> time timeuuid,
>>>>> PRIMARY KEY (initials, time)
>>>>> );
>>>>>
>>>>> insert into phone (initials, time, details) values ('su', now(),
>>>>> {'Sachi': '408-797-9600'});
>>>>> insert into phone(initials, time, details) values ('su', now(),
>>>>> {'Sam': '555-121-2222'});
>>>>>
>>>>>
>>>>> select * from phone;
>>>>>
>>>>>
>>>>> Results:
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> On Thu, Feb 4, 2016 at 11:55 PM, Atul Saroha <atul.saroha@snapdeal.com
>>>>> > wrote:
>>>>>
>>>>>> MySQL Table:
>>>>>> User | PhoneNumber
>>>>>> --------------------------------
>>>>>> raman    1234
>>>>>> bhuvan  2345
>>>>>> atul       5678
>>>>>>
>>>>>> Using single Collection column and map collection :
>>>>>>
>>>>>> Phone Number
>>>>>> ---------------------------
>>>>>> Map <raman,1234>
>>>>>> <bhuvan,2345>
>>>>>> <atul,5678>
>>>>>>
>>>>>> Want to transform data in this way, i.e. key is mapped from value
of
>>>>>> "user" column in the map.
>>>>>>
>>>>>> Any help will be appreciated.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------------------------------------------------------
>>>>>> Atul Saroha
>>>>>> *Sr. Software Engineer*
>>>>>> *M*: +91 8447784271 *T*: +91 124-415-6069 *EXT*: 12369
>>>>>> Plot # 362, ASF Centre - Tower A, Udyog Vihar,
>>>>>>  Phase -4, Sector 18, Gurgaon, Haryana 122016, INDIA
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Sachin S Uplaonkar.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Sachin S Uplaonkar.
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Sachin S Uplaonkar.
>>>
>>
>>
>

Mime
View raw message