Mailing-List: contact user-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@pig.apache.org
Received-SPF: pass (nike.apache.org: domain of royston.sellman@googlemail.com
 designates 74.125.82.177 as permitted sender)
Subject: Re: Cannot STORE from Pig to HBase 0.9x
References: <058001cce1eb$58447f70$08cd7e50$@gmail.com>
 <CAE7pYjbm6PFMhNPQ3P_smVjE+FAmwDFG+toYj==mrEJdd0dXUg@mail.gmail.com>
From: Royston Sellman <royston.sellman@googlemail.com>
Content-Type: text/plain;
	charset=us-ascii
In-Reply-To: 
 <CAE7pYjbm6PFMhNPQ3P_smVjE+FAmwDFG+toYj==mrEJdd0dXUg@mail.gmail.com>
Message-Id: <47E02ADC-70B0-4C8D-9DF0-B0C20EFC36B9@gmail.com>
Date: Thu, 2 Feb 2012 23:55:14 +0000
To: "user@pig.apache.org" <user@pig.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)

Thanks for your reply. WritableByteArrayComparable is in the same place in H=
Base 0.93. I also registered my HBase jar from within Pig i.e register /opt/=
hbase/hbase-0.93.jar and that command returned without error. I also edited t=
he line in the pig startup script that specifies the path to HBase.=20
Is there some other config file that controls how Pig finds HBase jar? Is th=
ere a default location Pig searches for jars that I could copy my jar into?

Cheers,
Royston


On 2 Feb 2012, at 23:38, Dmitriy Ryaboy <dvryaboy@gmail.com> wrote:

> "Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.filter.WritableByteArrayComparable" indicates that=

> you don't have HBase on your classpath, or that the version of HBase you
> are testing against moved this class someplace else. We've tested against
> the 0.90 series, but not 0.92+... did they refactor something?
>=20
> D
>=20
> On Thu, Feb 2, 2012 at 12:43 PM, Royston Sellman <
> royston.sellman@googlemail.com> wrote:
>=20
>> Hi,
>>=20
>>=20
>>=20
>> I'm trying to use Pig 0.9.2 with HBase 0.93 (i.e. the latest from HBase
>> trunk) and following the tutorial.
>>=20
>>=20
>>=20
>> This line loads the sample file from HDFS successfully:
>>=20
>> raw =3D LOAD 'test/excite-small.log' USING PigStorage('\t') AS (user, tim=
e,
>> query);
>>=20
>>=20
>>=20
>> This line seems to work:
>>=20
>> T=3D FOREACH raw GENERATE CONCAT(CONCAT(user, '\u0000'), time), query;
>>=20
>>=20
>>=20
>> Because when I do:
>>=20
>> DUMP T;
>>=20
>> I get the response I expected.
>>=20
>>=20
>>=20
>> But if I then do:
>>=20
>>               STORE T INTO 'excite' USING
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('colfam1:query');
>>=20
>> I get (in the log):
>>=20
>> Pig Stack Trace
>>=20
>> ---------------
>>=20
>> ERROR 2998: Unhandled internal error.
>> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>>=20
>>=20
>>=20
>> java.lang.NoClassDefFoundError:
>> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>>=20
>>       at java.lang.Class.forName0(Native Method)
>>=20
>>       at java.lang.Class.forName(Class.java:247)
>>=20
>>       at
>> org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:428)
>>=20
>>       at
>> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:45=
8)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuil=
der
>> .java:723)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder=
.ja
>> va:712)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerat=
or.
>> java:4340)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenera=
tor
>> .java:5956)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator=
.ja
>> va:1122)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanG=
ene
>> rator.java:683)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator=
.ja
>> va:483)
>>=20
>>       at
>>=20
>> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.jav=
a:3
>> 69)
>>=20
>>       at
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171)=

>>=20
>>       at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609=
)
>>=20
>>       at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582=
)
>>=20
>>       at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>=20
>>       at
>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
>>=20
>>       at
>>=20
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPars=
er.
>> java:386)
>>=20
>>       at
>>=20
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:=
188
>> )
>>=20
>>       at
>>=20
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:=
164
>> )
>>=20
>>       at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>=20
>>       at org.apache.pig.Main.run(Main.java:495)
>>=20
>>       at org.apache.pig.Main.main(Main.java:111)
>>=20
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>=20
>>       at
>>=20
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java=
:39
>> )
>>=20
>>       at
>>=20
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI=
mpl
>> .java:25)
>>=20
>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>=20
>>       at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>=20
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.hbase.filter.WritableByteArrayComparable
>>=20
>>       at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>=20
>>       at java.security.AccessController.doPrivileged(Native Method)
>>=20
>>       at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>=20
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>=20
>>       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>=20
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>=20
>>       ... 28 more
>>=20
>>=20
>>=20
>> Grateful for any help with this.
>>=20
>>=20
>>=20
>> Thanks,
>>=20
>> Royston
>>=20
>>=20