Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 875A811C75 for ; Wed, 23 Jul 2014 09:00:54 +0000 (UTC) Received: (qmail 42171 invoked by uid 500); 23 Jul 2014 09:00:51 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 42131 invoked by uid 500); 23 Jul 2014 09:00:51 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 42121 invoked by uid 99); 23 Jul 2014 09:00:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jul 2014 09:00:51 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of prvs=274da8090=akshay.ballarpure@tcs.com designates 121.242.48.3 as permitted sender) Received: from [121.242.48.3] (HELO inblrg01.tcs.com) (121.242.48.3) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jul 2014 09:00:47 +0000 X-IronPort-AV: E=Sophos;i="5.01,716,1400005800"; d="scan'208";a="115490292" Received: from INBLRDLPMTA1.india.tcs.com (unknown [127.0.0.1]) by INBLRDLPMTA1.india.tcs.com (Service) with ESMTP id BEA0FD2F14 for ; Wed, 23 Jul 2014 14:30:19 +0530 (IST) Received: from InHydM01.tcs.com (unknown [172.25.137.7]) by INBLRDLPMTA1.india.tcs.com (Service) with ESMTP id 89C93D2F02 for ; Wed, 23 Jul 2014 14:30:19 +0530 (IST) To: user@cassandra.apache.org MIME-Version: 1.0 Subject: CSV Import is taking huge time X-KeepSent: B3049B98:7E59F5B0-65257D1E:0030D156; type=4; name=$KeepSent X-Mailer: IBM Notes Release 9.0 March 08, 2013 Message-ID: From: Akshay Ballarpure Date: Wed, 23 Jul 2014 14:30:09 +0530 X-MIMETrack: Serialize by Notes Server on InHydM01/TCS(Release 9.0.1HF198 | January 23, 2014) at 07/23/2014 14:30:10, Serialize complete at 07/23/2014 14:30:10, Serialize by Router on InHydM01/TCS(Release 9.0.1HF198 | January 23, 2014) at 07/23/2014 14:30:10 Content-Type: multipart/alternative; boundary="=_alternative 00314E2C65257D1E_=" X-Virus-Checked: Checked by ClamAV on apache.org This is a multipart message in MIME format. --=_alternative 00314E2C65257D1E_= Content-Type: text/plain; charset="US-ASCII" Hello, I am trying copy command in Cassandra to import CSV file in to DB, Import is taking huge time, any suggestion to improve it? id,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z 100,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26 101,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26 ---- -- -- there are ~ 50 K lines in this file , size is ~ 5 MB. I have created table as per below: create table csldata4 ( id int PRIMARY KEY,a int , b int, c int, d int, e int, f int, g int, h int,i int, j int, k int, l int,m int, n int, o int, p int, q int, r int, s int, t int, u int, v int, w int, x int, y int , z int); Copy Command: COPY csldata4 (id , a , b , c , d , e , f , g , h , i , j , k , l , m , n , o , p , q , r , s , t , u , v , w , x , y , z ) FROM 'csldata1.csv' WITH HEADER=TRUE; Issue here is it's taking huge time to import cqlsh:mykeyspace> COPY csldata (id , a , b , c , d , e , f , g , h , i , j , k , l , m , n , o , p , q , r , s , t , u , v , w , x , y , z ) FROM 'csldata1.csv' WITH HEADER=TRUE; 66215 rows imported in 1 minute and 31.044 seconds. Thanks & Regards Akshay Ghanshyam Ballarpure Tata Consultancy Services Cell:- 9985084075 Mailto: akshay.ballarpure@tcs.com Website: http://www.tcs.com ____________________________________________ Experience certainty. IT Services Business Solutions Consulting ____________________________________________ =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you --=_alternative 00314E2C65257D1E_= Content-Type: text/html; charset="US-ASCII" Hello,
I am trying copy command in Cassandra to import CSV file in to DB, Import is taking huge time, any suggestion to improve it?

id,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
100,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26
101,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26
----
--
--

there are ~ 50 K lines in this file , size is ~ 5 MB.
 
I have created table as per below:

create table csldata4 ( id int PRIMARY KEY,a int , b int, c int, d int, e int, f int,
                        g int, h int,i int, j int, k int, l int,m int, n int, o int, p int, q                                         int, r int, s int, t int, u int, v int, w int, x int, y int , z int);
Copy Command:

COPY csldata4 (id , a , b , c , d , e , f , g , h , i , j , k , l , m , n , o , p , q , r , s , t , u , v , w , x , y , z ) FROM 'csldata1.csv' WITH HEADER=TRUE;
 
Issue here is it's taking huge time to import

cqlsh:mykeyspace> COPY csldata (id , a , b , c , d , e , f , g , h , i , j , k , l , m , n , o , p , q , r , s , t , u , v , w , x , y , z ) FROM 'csldata1.csv' WITH HEADER=TRUE;
66215 rows imported in 1 minute and 31.044 seconds.


Thanks & Regards
Akshay Ghanshyam Ballarpure
Tata Consultancy Services
Cell:- 9985084075
Mailto: akshay.ballarpure@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                       Business Solutions
                       Consulting
____________________________________________

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you

--=_alternative 00314E2C65257D1E_=--