Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 13 Jun 2016 11:13:20 +0000 (UTC)
From: "Rohith Sharma K S (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12969964.1463393193000.68898.1465816400968@Atlassian.JIRA>
In-Reply-To: <JIRA.12969964.1463393193000@Atlassian.JIRA>
References: <JIRA.12969964.1463393193000@Atlassian.JIRA> <JIRA.12969964.1463393193557@arcas>
Subject: [jira] [Commented] (YARN-5092) TestRMDelegationTokens fails
 intermittently
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 13 Jun 2016 11:13:22 -0000


    [ https://issues.apache.org/jira/browse/YARN-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327169#comment-15327169 ] 

Rohith Sharma K S commented on YARN-5092:
-----------------------------------------

Thanks [~jlowe] for  your analysis. I tested the patch changing the order of test case, i.e first {{testRemoveExpiredMasterKeyInRMStateStore}} and second {{testRMDTMasterKeyStateOnRollingMasterKey}}. Some issues are

# Test case {{testRMDTMasterKeyStateOnRollingMasterKey}} fails with MetricsException. It seems like clearing QueueMetrics at set up causing an issue. For smooth execution, In test *testRemoveExpiredMasterKeyInRMStateStore*, rm1 can be stopped at the end of the test cases by rm1.stop().{noformat}
org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root already exists!
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:230)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:131)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1418)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1479)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:658)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1001)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:290)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart$TestSecurityMockRM.init(TestRMRestart.java:2067)
	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:133)
	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:126)
	at org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart$TestSecurityMockRM.<init>(TestRMRestart.java:2060)
	at org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens$MyMockRM.<init>(TestRMDelegationTokens.java:168)
	at org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens.testRMDTMasterKeyStateOnRollingMasterKey(TestRMDelegationTokens.java:89)
{noformat}
# Continued after fixing first comment, still test case fails for kerberos authentication. Basically need to reset the login user either to null or pass a new ugi by {{UserGroupInformation.setLoginUser(null);}} at the start of test case. Otherwise, authentication method is taken from old ugi which is simple authentication.
{noformat}
org.apache.hadoop.yarn.exceptions.YarnException: java.io.IOException: Delegation Token can be issued only with kerberos authentication
	at org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getDelegationToken(ClientRMService.java:1069)
	at org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens.testRMDTMasterKeyStateOnRollingMasterKey(TestRMDelegationTokens.java:105)
	... 10 more
Caused by: java.io.IOException: Delegation Token can be issued only with kerberos authentication
	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getDelegationToken(ClientRMService.java:1042)
	... 10 more
{noformat}
# As you said, reason for ClassCastException is  one test runs with FS and other with CS. I think this can be removed and make generic to any schedulers which ever the scheduler configured default in the Jenkins. This will ensure test case pass independent of any schedulers.

> TestRMDelegationTokens fails intermittently 
> --------------------------------------------
>
>                 Key: YARN-5092
>                 URL: https://issues.apache.org/jira/browse/YARN-5092
>             Project: Hadoop YARN
>          Issue Type: Test
>          Components: test
>    Affects Versions: 2.7.2
>            Reporter: Rohith Sharma K S
>            Assignee: Jason Lowe
>         Attachments: YARN-5092.001.patch
>
>
> In build [link|https://builds.apache.org/job/PreCommit-YARN-Build/11476/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt] , TestRMDelegationTokens fails for 2 test cases
> # TestRMDelegationTokens.testRMDTMasterKeyStateOnRollingMasterKey
> # TestRMDelegationTokens.testRemoveExpiredMasterKeyInRMStateStore


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org