hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high
Date Fri, 29 Jun 2018 21:18:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528269#comment-16528269

Haibo Chen commented on YARN-6672:

Thanks [~elgoiri] for the review!
{quote} * In testPreemptOpportunisticContainersUponHighMemoryUtilization, can we force contianer
3 to be queued in the beginning and check that it's queued and not running? Then we check
utilization and let it run.{quote}
Good point. I added a check for container 3 to verify it is queued and not running, right
before the utilization is checked. It should take care of that.
{quote}In a more general scope (YARN-1011), there is one thing left that [~cheersyang] mentioned
offline: what happens when containers are starting and utilization is still low.

We need to create a utilization policy that is conservative and does not assign lots of load
to nodes with young containers.
Totally agree. This should be doable with another NMAllocationPolicy, but we have not gotten
to implement.
{quote} I think this will be there in YARN-7334; we may want to start iterating in that JIRA.
I'll start the documentation once we are done internally with the end-to-end workload-level
functional tests.


> Add NM preemption of opportunistic containers when utilization goes high
> ------------------------------------------------------------------------
>                 Key: YARN-6672
>                 URL: https://issues.apache.org/jira/browse/YARN-6672
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-6672-YARN-1011.00.patch, YARN-6672-YARN-1011.01.patch, YARN-6672-YARN-1011.02.patch,
YARN-6672-YARN-1011.03.patch, YARN-6672-YARN-1011.04.patch, YARN-6672-YARN-1011.05.patch,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message