pig-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Royston Sellman <royston.sell...@googlemail.com>
Subject Re: Cannot STORE from Pig to HBase 0.9x
Date Thu, 02 Feb 2012 23:55:14 GMT
Thanks for your reply. WritableByteArrayComparable is in the same place in HBase 0.93. I also
registered my HBase jar from within Pig i.e register /opt/hbase/hbase-0.93.jar and that command
returned without error. I also edited the line in the pig startup script that specifies the
path to HBase. 
Is there some other config file that controls how Pig finds HBase jar? Is there a default
location Pig searches for jars that I could copy my jar into?

Cheers,
Royston



On 2 Feb 2012, at 23:38, Dmitriy Ryaboy <dvryaboy@gmail.com> wrote:

> "Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.filter.WritableByteArrayComparable" indicates that
> you don't have HBase on your classpath, or that the version of HBase you
> are testing against moved this class someplace else. We've tested against
> the 0.90 series, but not 0.92+... did they refactor something?
> 
> D
> 
> On Thu, Feb 2, 2012 at 12:43 PM, Royston Sellman <
> royston.sellman@googlemail.com> wrote:
> 
>> Hi,
>> 
>> 
>> 
>> I'm trying to use Pig 0.9.2 with HBase 0.93 (i.e. the latest from HBase
>> trunk) and following the tutorial.
>> 
>> 
>> 
>> This line loads the sample file from HDFS successfully:
>> 
>> raw = LOAD 'test/excite-small.log' USING PigStorage('\t') AS (user, time,
>> query);
>> 
>> 
>> 
>> This line seems to work:
>> 
>> T= FOREACH raw GENERATE CONCAT(CONCAT(user, '\u0000'), time), query;
>> 
>> 
>> 
>> Because when I do:
>> 
>> DUMP T;
>> 
>> I get the response I expected.
>> 
>> 
>> 
>> But if I then do:
>> 
>>               STORE T INTO 'excite' USING
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('colfam1:query');
>> 
>> I get (in the log):
>> 
>> Pig Stack Trace
>> 
>> ---------------
>> 
>> ERROR 2998: Unhandled internal error.
>> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>> 
>> 
>> 
>> java.lang.NoClassDefFoundError:
>> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>> 
>>       at java.lang.Class.forName0(Native Method)
>> 
>>       at java.lang.Class.forName(Class.java:247)
>> 
>>       at
>> org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:428)
>> 
>>       at
>> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:458)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder
>> .java:723)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.ja
>> va:712)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.
>> java:4340)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator
>> .java:5956)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.ja
>> va:1122)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGene
>> rator.java:683)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.ja
>> va:483)
>> 
>>       at
>> 
>> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:3
>> 69)
>> 
>>       at
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171)
>> 
>>       at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>> 
>>       at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>> 
>>       at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>> 
>>       at
>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
>> 
>>       at
>> 
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.
>> java:386)
>> 
>>       at
>> 
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188
>> )
>> 
>>       at
>> 
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164
>> )
>> 
>>       at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>> 
>>       at org.apache.pig.Main.run(Main.java:495)
>> 
>>       at org.apache.pig.Main.main(Main.java:111)
>> 
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 
>>       at
>> 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
>> )
>> 
>>       at
>> 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
>> .java:25)
>> 
>>       at java.lang.reflect.Method.invoke(Method.java:597)
>> 
>>       at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> 
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.hbase.filter.WritableByteArrayComparable
>> 
>>       at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> 
>>       at java.security.AccessController.doPrivileged(Native Method)
>> 
>>       at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> 
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> 
>>       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> 
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> 
>>       ... 28 more
>> 
>> 
>> 
>> Grateful for any help with this.
>> 
>> 
>> 
>> Thanks,
>> 
>> Royston
>> 
>> 

Mime
View raw message