cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akshay Ballarpure <akshay.ballarp...@tcs.com>
Subject Cassandra - Pig integration
Date Thu, 31 Jul 2014 06:45:45 GMT
Hello,
I am trying to integrate cassandra into Hadoop and PIG and trying to load 
CSV file into Cassandra using PIG Script. Can someone help ?

root@hadoop-1:/home/hduser/apache-cassandra-2.0.9/examples/pig# cat 
pigCasandra.pig
data = LOAD 'example.csv' using PigStorage(',') AS (row_id: chararray, 
value1: chararray, value2: int);
data_to_insert = FOREACH data GENERATE TOTUPLE( TOTUPLE('row_id',row_id) 
), TOTUPLE(value1, value2);
STORE data_to_insert INTO 'cql://myschema/example?output_query=update 
example set value1 @ #,value2 @ #' USING CqlStorage();



root@hadoop-1:/home/hduser/apache-cassandra-2.0.9/examples/pig# 
/home/hduser/pig/pig-0.13.0/bin/pig pigCasandra.pig
14/07/31 17:38:00 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
14/07/31 17:38:00 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
14/07/31 17:38:00 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the 
ExecType
2014-07-31 17:38:00,078 [main] INFO  org.apache.pig.Main - Apache Pig 
version 0.13.0 (r1606446) compiled Jun 29 2014, 02:27:58
2014-07-31 17:38:00,078 [main] INFO  org.apache.pig.Main - Logging error 
messages to: 
/home/hduser/apache-cassandra-2.0.9/examples/pig/pig_1406808480077.log
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/home/hduser/yarn/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/home/hduser/apache-cassandra-2.0.9/lib/slf4j-log4j12-1.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library 
/home/hduser/yarn/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might 
have disabled sta       ck guard. The VM will try to fix the stack guard 
now.
It's highly recommended that you fix the library with 'execstack -c 
<libfile>', or link it with '-z noexecstack'.
2014-07-31 17:38:00,255 [main] WARN 
org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop 
library for your platform... using builtin-java c       lasses where 
applicable
2014-07-31 17:38:00,398 [main] INFO  org.apache.pig.impl.util.Utils - 
Default bootup file /root/.pigbootup not found
2014-07-31 17:38:00,484 [main] INFO 
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is 
deprecated. Instead, use mapreduce.jobtracker.a       ddress
2014-07-31 17:38:00,484 [main] INFO 
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is 
deprecated. Instead, use fs.defaultFS
2014-07-31 17:38:00,484 [main] INFO 
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - 
Connecting to hadoop file system at: hdfs://master:9000
2014-07-31 17:38:01,431 [main] INFO 
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is 
deprecated. Instead, use fs.defaultFS
2014-07-31 17:38:01,557 [main] INFO 
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is 
deprecated. Instead, use fs.defaultFS
2014-07-31 17:38:01,609 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
ERROR 2998: Unhandled internal error. 
com/datastax/driver/core/policies/LoadBalancing       Policy
Details at logfile: 
/home/hduser/apache-cassandra-2.0.9/examples/pig/pig_1406808480077.log

Thanks & Regards
Akshay Ghanshyam Ballarpure
Tata Consultancy Services
Cell:- 9985084075
Mailto: akshay.ballarpure@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty.   IT Services
                        Business Solutions
                        Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you



Mime
View raw message