crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Whitacre <mkw...@gmail.com>
Subject Re: Compilation Fails with Hadoop 2.5.1 and HBase 0.99.0
Date Sun, 28 Sep 2014 22:18:56 GMT
Dongjin,
I'm not sure we currently have any plans for setting Crunch up to do
conditional compilation but if there is enough need it could be explored.

With regard to upgrading to HBase 0.99, is there a specific need for that
version to be supported?  Since HBase treats "odd" releases[1] as
development, I wouldn't anticipate we would target releasing a version for
it.

[1] - http://hbase.apache.org/book/upgrading.html

On Sat, Sep 27, 2014 at 10:33 PM, Dongjin Lee <dongjin.lee.kr@gmail.com>
wrote:

> Hello. I am a developer who is working with hadoop 2.5.1.
> I tried to build crunch 0.11.0 with hbase 0.99.0, but it failed with
> following messages:
>
> [ERROR]
>
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[133,32]
> error: cannot find symbol
> [ERROR]
>   location: class KeyValue
>
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[146,33]
> error: incompatible types
> [ERROR]   required: KeyValue
>   found:    Cell
>
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[60,37]
> error: incompatible types
> [ERROR]   required: KeyValue
>   found:    Cell
>
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[73,36]
> error: incompatible types
> [ERROR]   required: KeyValue
>   found:    Cell
>
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:[422,28]
> error: cannot find symbol
> [INFO] 5 errors
>
> I found that this problem occurs by following issue:
> https://issues.apache.org/jira/browse/HBASE-11805 By this update, KeyValue
> class was replaced with Cell interface. It seems to have been fixed.
>
> If there is any problem, I would like to take the work. However, I have one
> question: Is there any plan to apply conditional compilation to Crunch,
> like Giraph? It uses maven munge plugin to achieve this[^1]. If there is, I
> think munge plugin and symbols should be applied first.
>
> Any guidance would be truly appreciated.
> Thanks in advance.
>
> [^1]: I already fixed giraph's compilation error problem, which occurs with
> hadoop 2.5.x. (https://issues.apache.org/jira/browse/GIRAPH-947)
>
> --
> *Dongjin Lee*
>
>
> *Oracle Server Technologies Group.So interested in massive-scale machine
> learning.facebook: www.facebook.com/dongjin.lee.kr
> <http://www.facebook.com/dongjin.lee.kr>linkedin:
> kr.linkedin.com/in/dongjinleekr
> <http://kr.linkedin.com/in/dongjinleekr>github:
> <http://goog_969573159>github.com/dongjinleekr
> <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
> <http://www.twitter.com/dongjinleekr>*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message