crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Whitacre <mkw...@gmail.com>
Subject Re: Compilation Fails with Hadoop 2.5.1 and HBase 0.99.0
Date Mon, 29 Sep 2014 02:43:55 GMT
A KeyValue is an implementation of the Cell interface.  From my experience
the Cell API has a similar API but doesn't have some of the methods which
KeyValue has and those KeyValue methods are deprecated.  Typically you have
to use the Cell + CellUtil API to get the same functionality as you have
with just KeyValue.

On Sun, Sep 28, 2014 at 8:47 PM, Josh Wills <jwills@cloudera.com> wrote:

> I don't mind conditional compilation on the assumption that we would keep
> it scoped to the hbase module; it seems like a lot of changes are coming
> down the pike in HBase land. Is Cell a 1:1 mapping of the functions of
> KeyValue?
>
> J
>
> On Sun, Sep 28, 2014 at 3:18 PM, Micah Whitacre <mkwhit@gmail.com> wrote:
>
> > Dongjin,
> > I'm not sure we currently have any plans for setting Crunch up to do
> > conditional compilation but if there is enough need it could be explored.
> >
> > With regard to upgrading to HBase 0.99, is there a specific need for that
> > version to be supported?  Since HBase treats "odd" releases[1] as
> > development, I wouldn't anticipate we would target releasing a version
> for
> > it.
> >
> > [1] - http://hbase.apache.org/book/upgrading.html
> >
> > On Sat, Sep 27, 2014 at 10:33 PM, Dongjin Lee <dongjin.lee.kr@gmail.com>
> > wrote:
> >
> > > Hello. I am a developer who is working with hadoop 2.5.1.
> > > I tried to build crunch 0.11.0 with hbase 0.99.0, but it failed with
> > > following messages:
> > >
> > > [ERROR]
> > >
> > >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[133,32]
> > > error: cannot find symbol
> > > [ERROR]
> > >   location: class KeyValue
> > >
> > >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileInputFormat.java:[146,33]
> > > error: incompatible types
> > > [ERROR]   required: KeyValue
> > >   found:    Cell
> > >
> > >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[60,37]
> > > error: incompatible types
> > > [ERROR]   required: KeyValue
> > >   found:    Cell
> > >
> > >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileReaderFactory.java:[73,36]
> > > error: incompatible types
> > > [ERROR]   required: KeyValue
> > >   found:    Cell
> > >
> > >
> >
> /home/dongjinleekr/workspace/hadoop/crunch/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:[422,28]
> > > error: cannot find symbol
> > > [INFO] 5 errors
> > >
> > > I found that this problem occurs by following issue:
> > > https://issues.apache.org/jira/browse/HBASE-11805 By this update,
> > KeyValue
> > > class was replaced with Cell interface. It seems to have been fixed.
> > >
> > > If there is any problem, I would like to take the work. However, I have
> > one
> > > question: Is there any plan to apply conditional compilation to Crunch,
> > > like Giraph? It uses maven munge plugin to achieve this[^1]. If there
> > is, I
> > > think munge plugin and symbols should be applied first.
> > >
> > > Any guidance would be truly appreciated.
> > > Thanks in advance.
> > >
> > > [^1]: I already fixed giraph's compilation error problem, which occurs
> > with
> > > hadoop 2.5.x. (https://issues.apache.org/jira/browse/GIRAPH-947)
> > >
> > > --
> > > *Dongjin Lee*
> > >
> > >
> > > *Oracle Server Technologies Group.So interested in massive-scale
> machine
> > > learning.facebook: www.facebook.com/dongjin.lee.kr
> > > <http://www.facebook.com/dongjin.lee.kr>linkedin:
> > > kr.linkedin.com/in/dongjinleekr
> > > <http://kr.linkedin.com/in/dongjinleekr>github:
> > > <http://goog_969573159>github.com/dongjinleekr
> > > <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
> > > <http://www.twitter.com/dongjinleekr>*
> > >
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message