hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2124) ProportionalCapacityPreemptionPolicy cannot work because it's initialized before scheduler initialized
Date Tue, 10 Jun 2014 03:08:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026080#comment-14026080
] 

Jian He commented on YARN-2124:
-------------------------------

Patch looks good overall.
Test case:  seems just starting RM with preemption enabled should be able to trigger the error?
if that, we can simplify the test case and don't need those surrounding changes only for test
purpose.

> ProportionalCapacityPreemptionPolicy cannot work because it's initialized before scheduler
initialized
> ------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2124
>                 URL: https://issues.apache.org/jira/browse/YARN-2124
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager, scheduler
>    Affects Versions: 3.0.0
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Critical
>         Attachments: YARN-2124.patch
>
>
> When I play with scheduler with preemption, I found ProportionalCapacityPreemptionPolicy
cannot work. NPE will be raised when RM start
> {code}
> 2014-06-05 11:01:33,201 ERROR org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
Thread[SchedulingMonitor (ProportionalCapacityPreemptionPolicy),5,main] threw an Exception.
> java.lang.NullPointerException
>         at org.apache.hadoop.yarn.util.resource.Resources.greaterThan(Resources.java:225)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.computeIdealResourceDistribution(ProportionalCapacityPreemptionPolicy.java:302)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.recursivelyComputeIdealAssignment(ProportionalCapacityPreemptionPolicy.java:261)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:198)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:174)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:72)
>         at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PreemptionChecker.run(SchedulingMonitor.java:82)
>         at java.lang.Thread.run(Thread.java:744)
> {code}
> This is caused by ProportionalCapacityPreemptionPolicy needs ResourceCalculator from
CapacityScheduler. But ProportionalCapacityPreemptionPolicy get initialized before CapacityScheduler
initialized. So ResourceCalculator will set to null in ProportionalCapacityPreemptionPolicy.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message