Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 10 Mar 2015 01:59:38 +0000 (UTC)
From: "Sean Busbey (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12779218.1425423566000.45482.1425952778928@Atlassian.JIRA>
In-Reply-To: <JIRA.12779218.1425423566000@Atlassian.JIRA>
References: <JIRA.12779218.1425423566000@Atlassian.JIRA>
 <JIRA.12779218.1425423566710@arcas>
Subject: [jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+
 Yarn
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354108#comment-14354108 ] 

Sean Busbey commented on HBASE-13149:
-------------------------------------

{quote}
Running Hbase with jackson 1.8 with Hadoop-2.5+ does not work at all, so I don't know how compatibility is in the picture.
{quote}

This is not true. While I think this issue shows a gap in our testing of the 1.0.0 release, I don't think that gap is large enough to say it doesn't work at all. So far as we know right now, HBase with a YARN based app (including our bundled MR tools) doesn't work at all. Downstream folks who don't care about running MR or other YARN apps against HBase should work just fine.

This is still a substantial problem, since it impacts folks who want to use our bundled tools, those who want to run MR directly, those who want to use Hive (atleast Hive on MR), Sqoop, and possibly Spark (in on-yarn mode).

The way I see it we have a few options that keeps us honest wrt our compatibility promises, in roughly order of my decreasing preference:

# Find a solution that fixes _just_ the problem at hand: running Hadoop 2.5+ YARN based apps against HBase
#* This might break operational compatibility, but we could do this without qualms on a minor version.
# Move branch-1 to be 2.0
#* We could fix this in branch-1, make it 2.0.0 and move master to 3.0.0. It would be simple and in line with our current promises. It would also do a great job of desensitizing us to major version number increments.
# Change our compatibility promise for dependencies to patch-only
#* I don't like this option because we just made our promises and I think not offering dependency compat at minor versions will be a substantial dis-incentive to upgrade
# Take a mulligan on jackson specifically
#* We could site this testing gap and the severity of the breakage in a big release note. But I really don't like this option because we just made our compatibility promises. It sets a dangerous precedent that erodes the trust downstream folks can place in our compatibility promises.


> HBase MR is broken on Hadoop 2.5+ Yarn
> --------------------------------------
>
>                 Key: HBASE-13149
>                 URL: https://issues.apache.org/jira/browse/HBASE-13149
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
>            Reporter: Jerry He
>            Priority: Critical
>         Attachments: HBASE-13149-0.98.patch, jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html, jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
>
>
> Running the server MR tools is not working on Yarn version 2.5+.
> Running org.apache.hadoop.hbase.mapreduce.Export:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
>         at org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
>         at org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:47)
>         at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
>         at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
>         at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
>         at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>         at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>         at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
> {noformat}
> The problem seems to be the jackson jar version.  HADOOP-10104 updated jackson version to 1.9.13.  YARN-2092 reported a problem as well.
> HBase is using jackson 1.8.8. This version of the jar in the classpath seem to cause the problem.
> Should we upgrade to jackson 1.9.13? 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)