hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5994) TestCapacityScheduler.testAMLimitUsage fails intermittently
Date Mon, 12 Dec 2016 19:10:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742822#comment-15742822
] 

Eric Badger commented on YARN-5994:
-----------------------------------

This test is racy and so the issue with it only manifests sometimes. YARN-5773 put in a change
so that {{verifyAMLimitForLeafQueue}} registers a 2GB node. {{queueA.getAMResourceLimit}}
should return 1024 for memory, but is racy and almost always gets set to 0 for memory. So
when the test submits the app with {{amResource2}}, that resource is set to 2048, instead
of 3072. In the instances that it gets correctly set to 3072, the test errors out that it
is trying to use more than the max memory available. 

The bandaid fix would be to increase the node size to 4GB, but that doesn't fix the race in
why {{queueA.getAMResourceLimit}} isn't getting updated before it is being accessed in the
test. 

[~wangda], [~varun_saxena], [~bibinchundatt], [~rohithsharma], [~Naganarasimha], [~sunilg]:
Does this analysis make sense? Should we just add a {{waitFor}} to wait until {{queueA.getAMResourceLimit}}
isn't 0,0? 

> TestCapacityScheduler.testAMLimitUsage fails intermittently
> -----------------------------------------------------------
>
>                 Key: YARN-5994
>                 URL: https://issues.apache.org/jira/browse/YARN-5994
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Eric Badger
>            Assignee: Eric Badger
>
> {noformat}
> java.lang.AssertionError: app shouldn't be null
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertNotNull(Assert.java:621)
> 	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForState(MockRM.java:169)
> 	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:577)
> 	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:488)
> 	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:395)
> 	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.verifyAMLimitForLeafQueue(TestCapacityScheduler.java:3389)
> 	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.testAMLimitUsage(TestCapacityScheduler.java:3251)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message