crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-475) Compilation problem caused by KeyValue -> Cell conversion
Date Tue, 14 Oct 2014 02:51:34 GMT


Micah Whitacre updated CRUNCH-475:
    Attachment: CRUNCH-475.patch

So since you brought this up on the mailing list I've been playing around with the code off
and on.  Thinking about this I personally see the progression path to HBase 1.0 as being...

1. Uplift current APIs to support Cell for reading/writing from HBase/HFile
2. Deprecate the KeyValue based APIs to encourage people to use Cell instead.
3. Refactor internals of Crunch (HFileUtil) to completely remove KeyValue when 1.0 has finally
been released.

By step #3, the upgrade path for consumers should be a lot simpler. 

Since HFile only supports KeyValue in HBase 0.98 it partially restricts how far down the Cell
migration we can go.  Avoiding playing games with compilation to keep the code clean and simple
serialization stayed compliant with KeyValue.  We'd obviously change it if HFile reading/writing
supported Cells.  In version 0.98 of HBase there are no Cell implementations besides KeyValue
so they are fairly intertwined.  

This patch is definitely a work in progress.  It currently only supports reading Cells.  I
haven't touched the code to support writing a Cell (still only writes through KeyValue). 
Additionally the patch is disruptive and doesn't support passivity or binary compatibility.

> Compilation problem caused by KeyValue -> Cell conversion
> ---------------------------------------------------------
>                 Key: CRUNCH-475
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.12.0
>            Reporter: Lee Dongjin
>            Assignee: Josh Wills
>            Priority: Minor
>         Attachments: CRUNCH-475.patch
> From hbase 0.99, Using KeyValue class for hbase I/O is deprecated and in many APIs it
was replaced with Cell interface[^1][^2][^3]. This change causes compilation error with hbase
0.99, which is the first hbase version that supports hadoop 2 only.
> For this change will be permanent from hbase 1.0 and on, it would be better to be fixed.
> [^1]:
> [^2]:
> [^3]:

This message was sent by Atlassian JIRA

View raw message