hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wilfred Spiegelenburg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9277) Add more restrictions In FairScheduler Preemption
Date Wed, 13 Feb 2019 06:03:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766811#comment-16766811

Wilfred Spiegelenburg commented on YARN-9277:

I agree with [~Steven Rand] sorting could be good but setting a hard no go could cause issues.

Can you also explain how we can pre-empt a container that is owned by the application itself?

I thought that we would only allow containers to be pre-empted if the application is over
its fair share and even then only if pre-empting the container would not drop the application
below its fair share. The {{FSPreemptionThread.identifyContainersToPreemptOnNode()}} calls
{{app.canContainerBePreempted()}} which contains that check and the container is not added.
Since the app we are pre-empting for is under its fair share any container of the app itself
should be filtered out by that. Am I reading this all wrong or have you found cases that we
did pre-empt a container for its own app and it is not working as expected?

> Add more restrictions In FairScheduler Preemption 
> --------------------------------------------------
>                 Key: YARN-9277
>                 URL: https://issues.apache.org/jira/browse/YARN-9277
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: fairscheduler
>            Reporter: Zhaohui Xin
>            Assignee: Zhaohui Xin
>            Priority: Major
>         Attachments: YARN-9277.001.patch, YARN-9277.002.patch
> I think we should add more restrictions in fair scheduler preemption. 
>  * We should not preempt self
>  * We should not preempt high priority job
>  * We should not preempt container which has been running for a long time.
>  * ...

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message