Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 65598 invoked from network); 13 Aug 2010 16:56:27 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 13 Aug 2010 16:56:27 -0000 Received: (qmail 98922 invoked by uid 500); 13 Aug 2010 16:56:25 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 98881 invoked by uid 500); 13 Aug 2010 16:56:25 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 98873 invoked by uid 99); 13 Aug 2010 16:56:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Aug 2010 16:56:25 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of stuhood@mailtrust.com designates 207.97.245.161 as permitted sender) Received: from [207.97.245.161] (HELO smtp161.iad.emailsrvr.com) (207.97.245.161) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Aug 2010 16:56:18 +0000 Received: from relay16.relay.iad.mlsrvr.com (localhost [127.0.0.1]) by relay16.relay.iad.mlsrvr.com (SMTP Server) with ESMTP id 0476F1B40B3 for ; Fri, 13 Aug 2010 12:55:58 -0400 (EDT) Received: from dynamic11.wm-web.iad.mlsrvr.com (dynamic11.wm-web.iad.mlsrvr.com [192.168.2.218]) by relay16.relay.iad.mlsrvr.com (SMTP Server) with ESMTP id F30441B408B for ; Fri, 13 Aug 2010 12:55:57 -0400 (EDT) Received: from mailtrust.com (localhost [127.0.0.1]) by dynamic11.wm-web.iad.mlsrvr.com (Postfix) with ESMTP id E463BE008A for ; Fri, 13 Aug 2010 12:55:57 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: stuhood@mailtrust.com, from: stu.hood@rackspace.com) with HTTP; Fri, 13 Aug 2010 11:55:57 -0500 (CDT) Date: Fri, 13 Aug 2010 11:55:57 -0500 (CDT) Subject: RE: Cassandra and Pig From: "Stu Hood" To: user@cassandra.apache.org MIME-Version: 1.0 Content-Type: text/plain;charset=UTF-8 Content-Transfer-Encoding: quoted-printable Importance: Normal X-Priority: 3 (Normal) X-Type: plain In-Reply-To: References: Message-ID: <1281718557.932823100@192.168.2.227> X-Mailer: webmail8 That error is coming from the frontend: the jars must also be on the local = classpath. Take a look at how contrib/pig/bin/pig_cassandra sets up $PIG_CL= ASSPATH.=0A=0A-----Original Message-----=0AFrom: "Christian Decker" =0ASent: Friday, August 13, 2010 11:30am=0ATo: user@ca= ssandra.apache.org=0ASubject: Cassandra and Pig=0A=0AHi all,=0A=0AI'm tryin= g to get Pig to read data from a Cassandra cluster, which I thought=0Atrivi= al since Cassandra already provides me with the CassandraStorage class.=0AP= roblem is that once I try executing a simple script like this:=0A=0Aregiste= r /path/to/pig-0.7.0-core.jar;register /path/to/libthrift-r917130.jar;=0Are= gister /path/to/cassandra_loadfunc.jarrows =3D LOAD=0A'cassandra://Keyspace= 1/Standard1' USING=0Aorg.apache.cassandra.hadoop.pig.CassandraStorage();col= s =3D FOREACH rows=0AGENERATE flatten($1);colnames =3D FOREACH cols GENERAT= E $0;namegroups =3D=0AGROUP colnames BY $0;namecounts =3D FOREACH namegroup= s GENERATE=0ACOUNT($1), group;orderednames =3D ORDER namecounts BY $0;topna= mes =3D=0ALIMIT orderednames 50;dump topnames;=0A=0AI just end up with a No= ClassDefFoundError:=0A=0AERROR org.apache.pig.tools.grunt.Grunt -=0Aorg.apa= che.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to=0Aopen i= terator for alias topnames=0Aat org.apache.pig.PigServer.openIterator(PigSe= rver.java:521)=0A at org.apache.pig.tools.grunt.GruntParser.processDump(Gru= ntParser.java:544)=0Aat=0Aorg.apache.pig.tools.pigscript.parser.PigScriptPa= rser.parse(PigScriptParser.java:241)=0A at=0Aorg.apache.pig.tools.grunt.Gru= ntParser.parseStopOnError(GruntParser.java:162)=0Aat=0Aorg.apache.pig.tools= .grunt.GruntParser.parseStopOnError(GruntParser.java:138)=0A at org.apache.= pig.tools.grunt.Grunt.exec(Grunt.java:89)=0Aat org.apache.pig.Main.main(Mai= n.java:391)=0ACaused by: org.apache.pig.impl.logicalLayer.FrontendException= : ERROR 1002:=0AUnable to store alias topnames=0A at org.apache.pig.PigServ= er.store(PigServer.java:577)=0Aat org.apache.pig.PigServer.openIterator(Pig= Server.java:504)=0A ... 6 more=0ACaused by: org.apache.pig.backend.executio= nengine.ExecException: ERROR 2117:=0AUnexpected error when launching map re= duce job.=0Aat=0Aorg.apache.pig.backend.hadoop.executionengine.mapReduceLay= er.MapReduceLauncher.launchPig(MapReduceLauncher.java:209)=0A at=0Aorg.apac= he.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEn= gine.java:308)=0Aat org.apache.pig.PigServer.executeCompiledLogicalPlan(Pig= Server.java:835)=0A at org.apache.pig.PigServer.store(PigServer.java:569)= =0A... 7 more=0ACaused by: java.lang.RuntimeException: Could not resolve er= ror that occured=0Awhen launching map reduce job: java.lang.NoClassDefFound= Error:=0Aorg/apache/thrift/TBase=0A at=0Aorg.apache.pig.backend.hadoop.exec= utionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandl= er.uncaughtException(MapReduceLauncher.java:510)=0A at java.lang.Thread.dis= patchUncaughtException(Thread.java:1845)=0A=0AI cannot think of a reason as= to why. As far as I understood it Pig takes=0Athe jar files in the script,= unpackages them, creates the execution plan for=0Athe script itself and th= en bundles it into a single jar again, then submits=0Ait to the HDFS from w= here it will be executed in Hadoop, right?=0AI also checked that the class = in question actually is in the libthrift jar,=0Aso what's going wrong?=0A= =0ARegards,=0AChris=0A