crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Kohn (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-549) crunch 0.12 not compatible with hbase 1.0
Date Wed, 22 Jul 2015 17:00:08 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tyler Kohn updated CRUNCH-549:
------------------------------
    Affects Version/s: 0.12.0

> crunch 0.12 not compatible with hbase 1.0
> -----------------------------------------
>
>                 Key: CRUNCH-549
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-549
>             Project: Crunch
>          Issue Type: Bug
>    Affects Versions: 0.12.0
>            Reporter: Tyler Kohn
>
> I'm using a patched (thanks - CRUNCH-540) version of 0.12 with CDH 5.4.3
> and getting:
> 2015-07-22 08:54:55,235 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running
child : java.lang.NoSuchMethodError: org.apache.hadoop.hbase.io.hfile.HFile$Writer.append(Lorg/apache/hadoop/hbase/KeyValue;)V
> at org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:102)
> at org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:95)
> at org.apache.crunch.io.CrunchOutputs$OutputState.write(CrunchOutputs.java:296)
> at org.apache.crunch.io.CrunchOutputs.write(CrunchOutputs.java:175)
> at org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
> at org.apache.crunch.MapFn.process(MapFn.java:34)
> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
> at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at org.apache.crunch.lib.PTables$1.process(PTables.java:163)
> at org.apache.crunch.lib.PTables$1.process(PTables.java:160)
> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
> at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at 
> looking at the code in 0.12, HFileOutputFormatForCrunch is 
> trying to write a KV (writer.append(kv);) into HFile.Writer
> HFile.Writer got rid of append(KeyValue kv) and now only supports Append(Cell cell) in
hbase 1.x.
> Am I doing something wrong?  I noticed that crunch0.12 compiles agains hbase 0.98.1 and
that version does have that interface still there.  if that's the case, is it possible to
get a cdh5 compatible version of crunch?
> thanks for the help.  not sure this is a *bug* necessarily but confused.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message