hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hu Ziqian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-8743) capacity scheduler doesn't set node label to reserved container
Date Tue, 04 Sep 2018 15:07:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hu Ziqian updated YARN-8743:
----------------------------
    Description: 
capacity scheduler doesn't set node label when new a reserved container's RMContainerImpl.
When allocate this container, leafQueue will treat it as a ignorePartitionExclusivityRMContainer.

It will cause preempt failure. When preempt happens, the preemption policy will try to preempt
the reserved container while leafQueue doesn't remove it from ignorePartitionExclusivityRMContainers.
In our cluster, we found that preemption policy will always try to preempt the reserved container
and actually preempt nothing.

We set the node label information to  reserved container's RMContainerImpl and redo our
test. The preemption performs as expected.

  was:
capacity scheduler doesn't set node label when new a reserved container's RMContainerImpl
and when allocate this container, leafQueue will treat it as a ignorePartitionExclusivityRMContainer.

This bug will cause preempt failure. When preempt happens, the preemption policy will try
to preempt the reserved container while leafQueue doesn't remove it from ignorePartitionExclusivityRMContainers.
In our cluster, we found that preemption policy will always try to preempt the reserved container
and actually preempt nothing.

We set the node label information to  reserved container's RMContainerImpl and redo our
test. The preemption performs as expected.


> capacity scheduler doesn't set node label to reserved container
> ---------------------------------------------------------------
>
>                 Key: YARN-8743
>                 URL: https://issues.apache.org/jira/browse/YARN-8743
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, scheduler preemption
>            Reporter: Hu Ziqian
>            Priority: Major
>
> capacity scheduler doesn't set node label when new a reserved container's RMContainerImpl.
When allocate this container, leafQueue will treat it as a ignorePartitionExclusivityRMContainer.
> It will cause preempt failure. When preempt happens, the preemption policy will try to
preempt the reserved container while leafQueue doesn't remove it from ignorePartitionExclusivityRMContainers.
In our cluster, we found that preemption policy will always try to preempt the reserved container
and actually preempt nothing.
> We set the node label information to  reserved container's RMContainerImpl and redo
our test. The preemption performs as expected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message