hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
Date Tue, 10 Mar 2015 01:59:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354108#comment-14354108

Sean Busbey commented on HBASE-13149:

Running Hbase with jackson 1.8 with Hadoop-2.5+ does not work at all, so I don't know how
compatibility is in the picture.

This is not true. While I think this issue shows a gap in our testing of the 1.0.0 release,
I don't think that gap is large enough to say it doesn't work at all. So far as we know right
now, HBase with a YARN based app (including our bundled MR tools) doesn't work at all. Downstream
folks who don't care about running MR or other YARN apps against HBase should work just fine.

This is still a substantial problem, since it impacts folks who want to use our bundled tools,
those who want to run MR directly, those who want to use Hive (atleast Hive on MR), Sqoop,
and possibly Spark (in on-yarn mode).

The way I see it we have a few options that keeps us honest wrt our compatibility promises,
in roughly order of my decreasing preference:

# Find a solution that fixes _just_ the problem at hand: running Hadoop 2.5+ YARN based apps
against HBase
#* This might break operational compatibility, but we could do this without qualms on a minor
# Move branch-1 to be 2.0
#* We could fix this in branch-1, make it 2.0.0 and move master to 3.0.0. It would be simple
and in line with our current promises. It would also do a great job of desensitizing us to
major version number increments.
# Change our compatibility promise for dependencies to patch-only
#* I don't like this option because we just made our promises and I think not offering dependency
compat at minor versions will be a substantial dis-incentive to upgrade
# Take a mulligan on jackson specifically
#* We could site this testing gap and the severity of the breakage in a big release note.
But I really don't like this option because we just made our compatibility promises. It sets
a dangerous precedent that erodes the trust downstream folks can place in our compatibility

> HBase MR is broken on Hadoop 2.5+ Yarn
> --------------------------------------
>                 Key: HBASE-13149
>                 URL: https://issues.apache.org/jira/browse/HBASE-13149
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0,
>            Reporter: Jerry He
>            Priority: Critical
>         Attachments: HBASE-13149-0.98.patch, jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
> Running the server MR tools is not working on Yarn version 2.5+.
> Running org.apache.hadoop.hbase.mapreduce.Export:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
>         at org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
>         at org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:47)
>         at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
>         at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
>         at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
>         at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>         at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>         at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
> {noformat}
> The problem seems to be the jackson jar version.  HADOOP-10104 updated jackson version
to 1.9.13.  YARN-2092 reported a problem as well.
> HBase is using jackson 1.8.8. This version of the jar in the classpath seem to cause
the problem.
> Should we upgrade to jackson 1.9.13? 

This message was sent by Atlassian JIRA

View raw message