incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood" <stu.h...@rackspace.com>
Subject RE: Cassandra and Pig
Date Fri, 13 Aug 2010 16:55:57 GMT
That error is coming from the frontend: the jars must also be on the local classpath. Take
a look at how contrib/pig/bin/pig_cassandra sets up $PIG_CLASSPATH.

-----Original Message-----
From: "Christian Decker" <decker.christian@gmail.com>
Sent: Friday, August 13, 2010 11:30am
To: user@cassandra.apache.org
Subject: Cassandra and Pig

Hi all,

I'm trying to get Pig to read data from a Cassandra cluster, which I thought
trivial since Cassandra already provides me with the CassandraStorage class.
Problem is that once I try executing a simple script like this:

register /path/to/pig-0.7.0-core.jar;register /path/to/libthrift-r917130.jar;
register /path/to/cassandra_loadfunc.jarrows = LOAD
'cassandra://Keyspace1/Standard1' USING
org.apache.cassandra.hadoop.pig.CassandraStorage();cols = FOREACH rows
GENERATE flatten($1);colnames = FOREACH cols GENERATE $0;namegroups =
GROUP colnames BY $0;namecounts = FOREACH namegroups GENERATE
COUNT($1), group;orderednames = ORDER namecounts BY $0;topnames =
LIMIT orderednames 50;dump topnames;

I just end up with a NoClassDefFoundError:

ERROR org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias topnames
at org.apache.pig.PigServer.openIterator(PigServer.java:521)
 at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
 at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
 at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:391)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
Unable to store alias topnames
 at org.apache.pig.PigServer.store(PigServer.java:577)
at org.apache.pig.PigServer.openIterator(PigServer.java:504)
 ... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2117:
Unexpected error when launching map reduce job.
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:209)
 at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:835)
 at org.apache.pig.PigServer.store(PigServer.java:569)
... 7 more
Caused by: java.lang.RuntimeException: Could not resolve error that occured
when launching map reduce job: java.lang.NoClassDefFoundError:
org/apache/thrift/TBase
 at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandler.uncaughtException(MapReduceLauncher.java:510)
 at java.lang.Thread.dispatchUncaughtException(Thread.java:1845)

I cannot think of a reason as to why. As far as I understood it Pig takes
the jar files in the script, unpackages them, creates the execution plan for
the script itself and then bundles it into a single jar again, then submits
it to the HDFS from where it will be executed in Hadoop, right?
I also checked that the class in question actually is in the libthrift jar,
so what's going wrong?

Regards,
Chris



Mime
View raw message