hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manu S <manupk...@gmail.com>
Subject Re: Pig + Hbase integration
Date Tue, 30 Oct 2012 05:27:38 GMT
Hi Jean,

This issue had been solved by following the suggestions of Cheolsoo

*1) ClassNotFoundError

Even though you're "registering" jars in your script, they're not present
in classpath. So you're seeing that ClassNotFound error. Can you try this?

PIG_CLASSPATH=<hbase_home>/hbase-0.94.1.jar:<hbase_home>/
lib/zookeeper-3.4.3.jar:<hbase_home>/lib/protobuf-java-2.4.0a.jar
./bin/pig <your script>

The best way to use HBaseStorage is to install the hbase client locally, so
they're present in classpath automatically. Then, you don't have to add
them to PIG_CLASSPATH.

2) pig-0.10.0.jar

Can you also make sure that you use pig-0.10.0-withouthadoop.jar instead of
pig-0.10.0.jar? Pig.jar embeds hbase-0.90, so you will run into
**a compatibly issue if you run it against hbase-0.94*.

Thanks,
Manu S
On Tue, Oct 30, 2012 at 4:25 AM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> On Thu, Oct 25, 2012 at 7:44 AM, Manu S <manupkd87@gmail.com> wrote:
> > Hi,
> >
> > I am using Pig-0.10.0 & hbase-0.94.2.
> >
> > I am trying to store the processed output to Hbase cluster using pig
> > script.
> >
> > I registered the required .jar and set the mapreduce and zookeeper
> > parameters within the script itself.
> >
> > *# cat input.pig*
> > register jar/hbase-0.94.2.jar;
> > register jar/zookeeper-3.4.3.jar;
> > register jar/protobuf-java-2.4.0a.jar;
> > register jar/guava-11.0.2.jar;
> > register jar/pig-0.10.0.jar;
> >
> > set fs.default.name hdfs://namenode:8020;
> > set mapred.job.tracker namenode:8021;
> > set hbase.cluster.distributed true;
> > set hbase.zookeeper.quorum namenode;
> > set hbase.master namenode:60000;
> > set hbase.zookeeper.property.clientPort 2181;
> > *
> > *
> > *raw_data = LOAD 'sample_data.csv' USING PigStorage( ',' ) AS (
> >  listing_id: chararray,fname: chararray,lname: chararray );*
> > *
> > *
> > *STORE raw_data INTO 'hbase://inputcsv' USING
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname
> info:lname');*
> >
> > When I execute the script I am getting this error
> >
> > *# pig input.pig*
> > *2012-10-25 19:55:08,331 [main] INFO  org.apache.pig.Main - Apache Pig
> > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12*
> > *2012-10-25 19:55:08,332 [main] INFO  org.apache.pig.Main - Logging error
> > messages to: /export/home/hadoop/devel/pig/pig_1351175108325.log*
> > *2012-10-25 19:55:08,944 [main] INFO
> >  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to hadoop file system at: hdfs://sangamt4:8020*
> > *2012-10-25 19:55:09,172 [main] INFO
> >  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to map-reduce job tracker at: sangamt4:8021*
> > *2012-10-25 19:55:10,021 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 2998: Unhandled internal error.
> > org/apache/hadoop/hbase/filter/WritableByteArrayComparable*
> > *Details at logfile: /export/home/hadoop/devel/pig/pig_1351175108325.log*
>
> And what are the details like in that log? Is it a classpath problem?
>
> J-D
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message