cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: read huge data from CSV and write into Cassandra
Date Fri, 25 Jul 2014 14:57:18 GMT
Read the csv file using a Java app and then index the rows using the Cassandra Java driver
with multiple, parallel input streams.

Oh, and make sure to provision your cluster with enough nodes to handle your desired ingestion
and query rates. Do a proof of concept with a six node cluster with RF=2 to see what ingestion
and query rates you can get for a fraction of your data and then scale from there. Although
a 12-node cluster with RF=3 would be more realistic. RF=2 is not for production – doesn’t
permit any failures, while RF=3 permits quorum operations with a single node failure. But
RF=2 at least lets you test with a more realistic scenario of coordinator nodes and inter-node
traffic.

And if your total row count does manage to fit on one machine (or three nodes with RF=3),
at least make sure you have enough CPU cores and I/O bandwidth to handle your desired ingestion
and query rate.

-- Jack Krupansky

From: Akshay Ballarpure 
Sent: Friday, July 25, 2014 5:26 AM
To: user@cassandra.apache.org 
Subject: read huge data from CSV and write into Cassandra

How to read data from large CSV file which is having 100+ columns and millions of rows and
inserting into Cassandra every 1 minute. 

Thanks & Regards
Akshay Ghanshyam Ballarpure
Tata Consultancy Services
Cell:- 9985084075
Mailto: akshay.ballarpure@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                       Business Solutions
                       Consulting
____________________________________________ 
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

Mime
View raw message