hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1444) RM crashes when node resource request sent without corresponding rack request
Date Tue, 11 Mar 2014 14:21:43 GMT

    [ https://issues.apache.org/jira/browse/YARN-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13930384#comment-13930384

Arun C Murthy commented on YARN-1444:

Btw, the description says 'without corresponding rack request', I think you mean 'off-switch

I agree that the NPE should be fixed, but the app will hang since there isn't an 'off-switch
request' and won't get any allocation... eventually, we need to detected this and fail the

> RM crashes when node resource request sent without corresponding rack request
> -----------------------------------------------------------------------------
>                 Key: YARN-1444
>                 URL: https://issues.apache.org/jira/browse/YARN-1444
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client, resourcemanager
>            Reporter: Robert Grandl
>            Assignee: Wangda Tan
>            Priority: Blocker
>         Attachments: yarn-1444.ver1.patch
> I have tried to force reducers to execute on certain nodes. What I did is I changed for
reduce tasks, the RMContainerRequestor#addResourceRequest(req.priority, ResourceRequest.ANY,
req.capability) to RMContainerRequestor#addResourceRequest(req.priority, HOST_NAME, req.capability).

> However, this change lead to RM crashes when reducers needs to be assigned with the following
> FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling
event type NODE_UPDATE to the scheduler
> java.lang.NullPointerException
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:841)
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:640)
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:554)
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:695)
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:739)
>     at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:86)
>     at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:549)
>     at java.lang.Thread.run(Thread.java:722)

This message was sent by Atlassian JIRA

View raw message