hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6081) LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container
Date Wed, 11 Jan 2017 03:07:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816990#comment-15816990
] 

Wangda Tan commented on YARN-6081:
----------------------------------

This is the test case to reproduce the problem:

{code}
  @Test
  public void testPreemptionNotHappenForSingleReservedQueue() {
    Logger rootLogger = LogManager.getRootLogger();
    rootLogger.setLevel(Level.DEBUG);

    int[][] qData = new int[][]{
        //  /   A   B   C
        { 100, 40, 40, 20 },  // abs
        { 100, 100, 100, 100 },  // maxCap
        { 100,  70,  0,  0 },  // used
        {  10, 30,  0,  0 },  // pending
        {   0,  50,  0,  0 },  // reserved
        {   1,  1,  0,  0 },  // apps
        {  -1,  1,  1,  1 },  // req granularity
        {   3,  0,  0,  0 },  // subqueues
    };
    ProportionalCapacityPreemptionPolicy policy = buildPolicy(qData);
    policy.editSchedule();
    // ensure all pending rsrc from A get preempted from other queues
    verify(mDisp, times(0)).handle(argThat(new IsPreemptionRequestFor(appA)));
  }
{code}

Please note that there's only one active queue. But preemption policy still preempt container
from it.

> LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending
to avoid unnecessary preemption of reserved container
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6081
>                 URL: https://issues.apache.org/jira/browse/YARN-6081
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Critical
>
> While doing YARN-5864 tests, found an issue when a queue's reserved > pending. PreemptionResourceCalculator
will preempt reserved container even if there's only one active queue in the cluster. 
> To fix the problem, we need to deduct reserved from pending when getting total-pending
resource for LeafQueue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message