cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anuj Wadehra <anujw_2...@yahoo.co.in>
Subject Re: Moving Away from Compact Storage
Date Wed, 03 Feb 2016 03:11:34 GMT
By dynamic columns, I mean columns not defined in schema. In current scenario, every row has
some data in columns which are defined in schema while rest of the data is in columns which
are not defined in schema. We used Thrift for inserting data.
In new schema, we want to create a collection column and put all the data which was there
in columns NOT defined in schema to the collection. 

ThanksAnuj

Sent from Yahoo Mail on Android 
 
  On Wed, 3 Feb, 2016 at 12:36 am, DuyHai Doan<doanduyhai@gmail.com> wrote:   You 'll
need to do the transformation in Spark, although I don't understand what you mean by "dynamic
columns". Given the CREATE TABLE script you gave earlier, there is nothing such as dynamic
columns
On Tue, Feb 2, 2016 at 8:01 PM, Anuj Wadehra <anujw_2003@yahoo.co.in> wrote:

Will it be possible to read dynamic columns data from compact storage and trasform them as
collection e.g. map in new table?

ThanksAnuj

Sent from Yahoo Mail on Android 
 
 On Wed, 3 Feb, 2016 at 12:28 am, DuyHai Doan<doanduyhai@gmail.com> wrote:   So there
is no "static" (in the sense of CQL static) column in your legacy table. 
Just define a Scala case class to match this table and use Spark to dump the content to a
new non compact CQL table
On Tue, Feb 2, 2016 at 7:55 PM, Anuj Wadehra <anujw_2003@yahoo.co.in> wrote:

Our old table looks like this from cqlsh:
CREATE TABLE table table1 (  key text,  "Col1" blob,  "Col2" text,  "Col3" text,  "Col4"
text,  PRIMARY KEY (key)) WITH COMPACT STORAGE AND …
And it will have some dynamic text data which we are planning to add in collections..
Please let me know if you need more details..

ThanksAnujSent from Yahoo Mail on Android 
 
 On Wed, 3 Feb, 2016 at 12:14 am, DuyHai Doan<doanduyhai@gmail.com> wrote:   Can you
give the CREATE TABLE script for you old compact storage table ? Or at least the cassandra-client
creation script
On Tue, Feb 2, 2016 at 3:48 PM, Anuj Wadehra <anujw_2003@yahoo.co.in> wrote:

Thanks DuyHai !! We were also thinking to do it the "Spark" way but I was not sure that its
would be so simple :)

We have a compact storage cf with each row having some data in staticly defined columns while
other data in dynamic columns. Is the approach mentioned in link adaptable to the scenario
where we want to migrate the existing data to a Non-Compact CF with static columns and collections
?

Thanks
Anuj

--------------------------------------------
On Tue, 2/2/16, DuyHai Doan <doanduyhai@gmail.com> wrote:

 Subject: Re: Moving Away from Compact Storage
 To: user@cassandra.apache.org
 Date: Tuesday, 2 February, 2016, 12:57 AM

 Use Apache
 Spark to parallelize the data migration. Look at this piece
 of code https://github.com/doanduyhai/Cassandra-Spark-Demo/blob/master/src/main/scala/usecases/MigrateAlbumsData.scala#L58-L60
 If your source and target tables
 have the SAME structure (except for the COMPACT STORAGE
 clause), migration with Spark is a 2 lines of
 code
 On Mon, Feb 1, 2016 at 8:14
 PM, Anuj Wadehra <anujw_2003@yahoo.co.in>
 wrote:
 Hi
 Whats the fastest and reliable way
 to migrate data from a Compact Storage table to Non-Compact
 storage table?
 I was not
 able to find any command for dropping the compact storage
 directive..so I think migrating data is the only way...any
 suggestions?
 ThanksAnuj




  


  


  

Mime
View raw message