Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 22 Nov 2017 06:13:00 +0000 (UTC)
From: "Rohith Sharma K S (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13120060.1511297692000.296404.1511331180300@Atlassian.JIRA>
In-Reply-To: <JIRA.13120060.1511297692000@Atlassian.JIRA>
References: <JIRA.13120060.1511297692000@Atlassian.JIRA> <JIRA.13120060.1511297692448@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-15059) 3.0 deployment cannot work with
 old version MR tar ball which break rolling upgrade
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 22 Nov 2017 06:13:05 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-15059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262023#comment-16262023 ] 

Rohith Sharma K S commented on HADOOP-15059:
--------------------------------------------

Atsv2 officially claims HBase-1.2.6 as backend. It works _absolutely fine_ in non-secure mode i.e installing *Hadoop-3.0 + HBase-1.2.6*. 
But the same deployment in secured cluster does not work because HBase-1.2.6 does not communicate to Hadoop-3.x because of token proto mismatch. Basically HMaster daemon start fails with exception while it is connecting into Hadoop-3.x in secure cluster!

To simplify the problem, Hadoop-2.x clients(HBase-1.2.6 compiled against Hadoop-2.x) doesn't communicate with Hadoop-3.x cluster. Are we going to keep binary compatibility across hadoop-2.x and hadoop-3.x? Similar scenario can happen while rolling upgrade as well which reported in this JIRA. 

Btw, from ATSv2 we are planning to add this in documentation as known issue until hbase release 2.x. cc:/[~vrushalic] [~varun_saxena]

> 3.0 deployment cannot work with old version MR tar ball which break rolling upgrade
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-15059
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15059
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: security
>            Reporter: Junping Du
>            Priority: Blocker
>
> I tried to deploy 3.0 cluster with 2.9 MR tar ball. The MR job is failed because following error:
> {noformat}
> 2017-11-21 12:42:50,911 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1511295641738_0003_000001
> 2017-11-21 12:42:51,070 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 2017-11-21 12:42:51,118 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> java.lang.RuntimeException: Unable to determine current user
> 	at org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:254)
> 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:220)
> 	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:212)
> 	at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:888)
> 	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1638)
> Caused by: java.io.IOException: Exception reading /tmp/nm-local-dir/usercache/jdu/appcache/application_1511295641738_0003/container_e03_1511295641738_0003_01_000001/container_tokens
> 	at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:208)
> 	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:907)
> 	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:820)
> 	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:689)
> 	at org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:252)
> 	... 4 more
> Caused by: java.io.IOException: Unknown version 1 in token storage.
> 	at org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:226)
> 	at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:205)
> 	... 8 more
> 2017-11-21 12:42:51,122 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.lang.RuntimeException: Unable to determine current user
> {noformat}
> I think it is due to token incompatiblity change between 2.9 and 3.0. As we claim "rolling upgrade" is supported in Hadoop 3, we should fix this before we ship 3.0 otherwise all MR running applications will get stuck during/after upgrade.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org