crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Kohn (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-549) crunch 0.12 not compatible with hbase 1.0
Date Wed, 22 Jul 2015 16:38:04 GMT
Tyler Kohn created CRUNCH-549:
---------------------------------

             Summary: crunch 0.12 not compatible with hbase 1.0
                 Key: CRUNCH-549
                 URL: https://issues.apache.org/jira/browse/CRUNCH-549
             Project: Crunch
          Issue Type: Bug
            Reporter: Tyler Kohn



I'm using a patched (thanks - CRUNCH-540) version of 0.12 with CDH 5.4.3

and getting:

2015-07-22 08:54:55,235 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child
: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.io.hfile.HFile$Writer.append(Lorg/apache/hadoop/hbase/KeyValue;)V
at org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:102)
at org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:95)
at org.apache.crunch.io.CrunchOutputs$OutputState.write(CrunchOutputs.java:296)
at org.apache.crunch.io.CrunchOutputs.write(CrunchOutputs.java:175)
at org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
at org.apache.crunch.MapFn.process(MapFn.java:34)
at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
at org.apache.crunch.lib.PTables$1.process(PTables.java:163)
at org.apache.crunch.lib.PTables$1.process(PTables.java:160)
at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
at 

looking at the code in 0.12, HFileOutputFormatForCrunch is 

trying to write a KV (writer.append(kv);) into HFile.Writer

HFile.Writer got rid of append(KeyValue kv) and now only supports Append(Cell cell) in hbase
1.x.

Am I doing something wrong?  I noticed that crunch0.12 compiles agains hbase 0.98.1 and that
version does have that interface still there.  if that's the case, is it possible to get a
cdh5 compatible version of crunch?

thanks for the help.  not sure this is a *bug* necessarily but confused.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message