hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
Date Tue, 10 Mar 2015 18:14:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355362#comment-14355362

Enis Soztutar commented on HBASE-13149:

bq. Our compatibility guide currently says we will only cause a breaking dependency change
on major version. The comparison Jerry He did above shows that Jackson 1.8 -> Jackson 1.9
is a breaking dependency change, unless we're missing something in the jackson library that
scopes which packages / classes should be relied on by downstream clients.
I think the changes are in the internals of Jackson from what I see in the report for jackson-core.
I do not realistically expect us to do a full sweep of every dependency compat story, so we
should rely on what our dependencies use for versioning. Assuming that jackson (or other deps)
uses semver or something similar going from 1.8 to 1.9 should be fine. 
I do not think we should corner ourselves into "cannot upgrade dependencies" situation (I
think [~ndimiduk] started a thread on this some time ago). From what I see, inconveniencing
our users with not upgrading this lib in 1.1 is far worse than upgrading it, but possibly
causing some minor (if any) inconvenience in case the user is relying on that lib explicitly.

> HBase MR is broken on Hadoop 2.5+ Yarn
> --------------------------------------
>                 Key: HBASE-13149
>                 URL: https://issues.apache.org/jira/browse/HBASE-13149
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0,
>            Reporter: Jerry He
>            Priority: Critical
>         Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch, jackson-core-asl-compat_report.html,
jackson-jaxrs-compat_report.html, jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
> Running the server MR tools is not working on Yarn version 2.5+.
> Running org.apache.hadoop.hbase.mapreduce.Export:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
>         at org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
>         at org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:47)
>         at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
>         at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>         at org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
>         at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
>         at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
>         at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
>         at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
>         at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>         at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>         at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
> {noformat}
> The problem seems to be the jackson jar version.  HADOOP-10104 updated jackson version
to 1.9.13.  YARN-2092 reported a problem as well.
> HBase is using jackson 1.8.8. This version of the jar in the classpath seem to cause
the problem.
> Should we upgrade to jackson 1.9.13? 

This message was sent by Atlassian JIRA

View raw message