hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high
Date Fri, 29 Jun 2018 21:18:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528269#comment-16528269
] 

Haibo Chen commented on YARN-6672:
----------------------------------

Thanks [~elgoiri] for the review!
{quote} * In testPreemptOpportunisticContainersUponHighMemoryUtilization, can we force contianer
3 to be queued in the beginning and check that it's queued and not running? Then we check
utilization and let it run.{quote}
Good point. I added a check for container 3 to verify it is queued and not running, right
before the utilization is checked. It should take care of that.
{quote}In a more general scope (YARN-1011), there is one thing left that [~cheersyang] mentioned
offline: what happens when containers are starting and utilization is still low.

We need to create a utilization policy that is conservative and does not assign lots of load
to nodes with young containers.
{quote}
Totally agree. This should be doable with another NMAllocationPolicy, but we have not gotten
to implement.
{quote} I think this will be there in YARN-7334; we may want to start iterating in that JIRA.
{quote}
I'll start the documentation once we are done internally with the end-to-end workload-level
functional tests.

 

> Add NM preemption of opportunistic containers when utilization goes high
> ------------------------------------------------------------------------
>
>                 Key: YARN-6672
>                 URL: https://issues.apache.org/jira/browse/YARN-6672
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-6672-YARN-1011.00.patch, YARN-6672-YARN-1011.01.patch, YARN-6672-YARN-1011.02.patch,
YARN-6672-YARN-1011.03.patch, YARN-6672-YARN-1011.04.patch, YARN-6672-YARN-1011.05.patch,
YARN-6672-YARN-1011.06.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message