crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-626) HBaseSourceTarget Binary compatibility across HBase versions
Date Thu, 01 Dec 2016 09:07:58 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711412#comment-15711412
] 

Tom White commented on CRUNCH-626:
----------------------------------

It doesn't affect this issue, but FYI the HBase APIs are changing again in HBase 2, so there'll
be more work to support that (particularly HFile) when it's generally available. See CRUNCH-619.

> HBaseSourceTarget Binary compatibility across HBase versions
> ------------------------------------------------------------
>
>                 Key: CRUNCH-626
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-626
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>         Attachments: CRUNCH-626.patch
>
>
> Currently Crunch targets HBase 1.0.   Certain vendors however are trying to maintain
API compatibility with HBase 0.96, 0.98, 1.0, and all 1.x.  
> While the API compatibility is ok across these versions, it was however not binary compatible
as HBase changed the method signature from returning void[1] to returning OperationWithAttributes[1]
(and in later versions Scan).
> This results in an exception like the following, when you try to use the latest Crunch
0.14 release against that vendor's distribution of HBase which we should be theoretically
compatible with.
> {noformat}
> java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/Scan;
> at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:112)
> at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:96)
> at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:88)
> {noformat}
> [1] - https://github.com/apache/hbase/blob/rel/0.98.0/hbase-client/src/main/java/org/apache/hadoop/hbase/client/OperationWithAttributes.java#L40
> [2] - https://github.com/apache/hbase/blob/rel/1.0.0/hbase-client/src/main/java/org/apache/hadoop/hbase/client/OperationWithAttributes.java#L41



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message