hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8511) When AM releases a container, RM removes allocation tags before it is released by NM
Date Fri, 13 Jul 2018 03:05:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542446#comment-16542446

Weiwei Yang commented on YARN-8511:

Hi [~leftnoteasy]
{quote}Instead of doing this, is it better to pass RMContext to SchedulerNode, so schedulerNode
can directly access RMContext like SchedulerAppAttempt?
We don't need to pass RMContext to SchedulerNode, because SchedulerNode is constructed with
RMNode, and RMNode has a reference to RMContext (but private). An alternative approach is
to expose RM context through RMNode interface so scheduler node could access it. This still
needs to modify RMNode and its sub-classes, but this is a once-for-all change, next time if
we want to call something different in the context inside of SchedulerNode, it will be straightforward.

Please let me know if v3 patch looks good to you, thanks!

> When AM releases a container, RM removes allocation tags before it is released by NM
> ------------------------------------------------------------------------------------
>                 Key: YARN-8511
>                 URL: https://issues.apache.org/jira/browse/YARN-8511
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>    Affects Versions: 3.1.0
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: YARN-8511.001.patch, YARN-8511.002.patch
> User leverages PC with allocation tags to avoid port conflicts between apps, we found
sometimes they still get port conflicts. This is a similar issue like YARN-4148. Because RM
immediately removes allocation tags once AM#allocate asks to release a container, however
container on NM has some delay until it actually gets killed and released the port. We should
let RM remove allocation tags AFTER NM confirms the containers are released. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message