hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gergely Novák (JIRA) <j...@apache.org>
Subject [jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
Date Wed, 20 Jul 2016 09:30:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385594#comment-15385594
] 

Gergely Novák commented on YARN-4954:
-------------------------------------

This JIRA seems outdated to me: unfortunately I'm not familiar with the latest developments
affecting the YARN reservation system, but it looks like the MiniYARNCluster is now using
hard coded config values for 'detecting' container memory (NodeManagerHardwareUtils.getContainerMemoryMB()):
{code}
memoryMb = conf.getInt(YarnConfiguration.NM_PMEM_MB, YarnConfiguration.DEFAULT_NM_PMEM_MB);
{code}

This means that it is irrelevant how much memory is actually available in the machine, I tested
this with VMs containing 6, 4, 3 and 2 GBs of RAM (with swapping disabled) and on my local
machine with 16 GBs: the container memory is *always* 4096 MBs, and the plan capacity is 4096
* 0.8 * 2 \[total * planAbsCap * number of node managers\] = 6553 MBs, so the block you quoted
works (at least in my test environments). Also, with fixed container memory the fixed reservation
request memory sizes seem reasonable to me.

[~vinodkv] Can you give me a link to the environment (Jenkins?) where the tests are consistently
failing so that I can have a further analysis?

> TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
> -----------------------------------------------------------------------------------------
>
>                 Key: YARN-4954
>                 URL: https://issues.apache.org/jira/browse/YARN-4954
>             Project: Hadoop YARN
>          Issue Type: Test
>          Components: test
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Gergely Novák
>            Assignee: Gergely Novák
>            Priority: Critical
>         Attachments: YARN-4954.001.patch
>
>
> TestYarnClient.testReservationAPIs sometimes fails with this error:
> {noformat}
> java.lang.AssertionError: org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
The request cannot be satisfied
> 	at org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
> 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
> 	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
> 	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
> Caused by: org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
The request cannot be satisfied
> 	at org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
> 	at org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
> 	at org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
> 	at org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
> 	at org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
> 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
> 	... 10 more
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
> {noformat}
> This is caused by really not having enough available memory to complete the reservation
(4 * 1024 MB). In my opinion lowering the required memory (either by lowering the number of
containers to 2, or the memory to 512 MB) would make the test more stable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message