crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Compilation Fails with Hadoop 2.5.1 and HBase 0.99.0
Date Mon, 29 Sep 2014 01:47:19 GMT
I don't mind conditional compilation on the assumption that we would keep
it scoped to the hbase module; it seems like a lot of changes are coming
down the pike in HBase land. Is Cell a 1:1 mapping of the functions of
KeyValue?

J

On Sun, Sep 28, 2014 at 3:18 PM, Micah Whitacre <mkwhit@gmail.com> wrote:

> Dongjin,
> I'm not sure we currently have any plans for setting Crunch up to do
> conditional compilation but if there is enough need it could be explored.
>
> With regard to upgrading to HBase 0.99, is there a specific need for that
> version to be supported?  Since HBase treats "odd" releases[1] as
> development, I wouldn't anticipate we would target releasing a version for
> it.
>
> [1] - http://hbase.apache.org/book/upgrading.html
>
> On Sat, Sep 27, 2014 at 10:33 PM, Dongjin Lee <dongjin.lee.kr@gmail.com>
> wrote:
>
> > Hello. I am a developer who is working with hadoop 2.5.1.
> > I tried to build crunch 0.11.0 with hbase 0.99.0, but it failed with
> > following messages:
> >
> > [ERROR]
> >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[133,32]
> > error: cannot find symbol
> > [ERROR]
> >   location: class KeyValue
> >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[146,33]
> > error: incompatible types
> > [ERROR]   required: KeyValue
> >   found:    Cell
> >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[60,37]
> > error: incompatible types
> > [ERROR]   required: KeyValue
> >   found:    Cell
> >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[73,36]
> > error: incompatible types
> > [ERROR]   required: KeyValue
> >   found:    Cell
> >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:[422,28]
> > error: cannot find symbol
> > [INFO] 5 errors
> >
> > I found that this problem occurs by following issue:
> > https://issues.apache.org/jira/browse/HBASE-11805 By this update,
> KeyValue
> > class was replaced with Cell interface. It seems to have been fixed.
> >
> > If there is any problem, I would like to take the work. However, I have
> one
> > question: Is there any plan to apply conditional compilation to Crunch,
> > like Giraph? It uses maven munge plugin to achieve this[^1]. If there
> is, I
> > think munge plugin and symbols should be applied first.
> >
> > Any guidance would be truly appreciated.
> > Thanks in advance.
> >
> > [^1]: I already fixed giraph's compilation error problem, which occurs
> with
> > hadoop 2.5.x. (https://issues.apache.org/jira/browse/GIRAPH-947)
> >
> > --
> > *Dongjin Lee*
> >
> >
> > *Oracle Server Technologies Group.So interested in massive-scale machine
> > learning.facebook: www.facebook.com/dongjin.lee.kr
> > <http://www.facebook.com/dongjin.lee.kr>linkedin:
> > kr.linkedin.com/in/dongjinleekr
> > <http://kr.linkedin.com/in/dongjinleekr>github:
> > <http://goog_969573159>github.com/dongjinleekr
> > <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
> > <http://www.twitter.com/dongjinleekr>*
> >
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message