pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roman Shaposhnik (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2220) Pig/HBase integration could be improved
Date Tue, 16 Aug 2011 20:54:27 GMT

    [ https://issues.apache.org/jira/browse/PIG-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085969#comment-13085969
] 

Roman Shaposhnik commented on PIG-2220:
---------------------------------------

First of all, thanks for pointing out that Pig (0.8.1+) already tries to do that.
However, it doesn't quite seem to work for me. Even the simplest query:

{noformat}
store data into 'rvs-tbl' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('family1:cnt1
family1:cnt2 family2:name', '-caster HBaseBinaryConverter');
{noformat}

generates class not found:

{noformat}
Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.filter.WritableByteArrayComparable
{noformat}

Registering hbase JAR manually fixes the problem. So perhaps, there's a bug in there somewhere.
Or may be I'm not
fully following your comment. Please elaborate.

> Pig/HBase integration could be improved
> ---------------------------------------
>
>                 Key: PIG-2220
>                 URL: https://issues.apache.org/jira/browse/PIG-2220
>             Project: Pig
>          Issue Type: Improvement
>          Components: grunt
>    Affects Versions: 0.9.0, 0.10
>            Reporter: Roman Shaposhnik
>            Priority: Minor
>
> For now any Pig/HBase queries would require the following jars to be explicitly added
via Pig's register command:
> register /usr/lib/hbase/lib/zookeeper-3.3.3-cdh3u1.jar; 
> register /usr/lib/hbase/hbase-0.90.3-cdh3u1.jar; 
> register /usr/lib/hbase/lib/guava-r06.jar;
> the longer term solution, perhaps, should be to have the code at submit time call hbase's

> TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship it in distributedcache.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message