hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Botong Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6704) Add Federation Interceptor restart when work preserving NM is enabled
Date Sat, 02 Dec 2017 01:43:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275356#comment-16275356
] 

Botong Huang commented on YARN-6704:
------------------------------------

Thanks [~subru] for the comments! In v6 patch, I've changed the UAM token storage to use local
NMSS instead when AMRMProxy HA is not enabled. 

NMSS cleanup is done by {{FederationInterceptor}} normally. Furthermore there's another safeguard
in {{AMRMProxy}}. Whenever the interceptor pipeline for an app is destroyed, {{AMRMProxy}}
will delete all NMSS entries associated with the app. For registry cleanup, we only do it
if _finishApplicationMaster_ is called and all UAMs unregister successfully. Otherwise, we
cannot delete the entries because there might be a second attempt for it. The cleanup service
will be introduced in YARN-7599. 

> Add Federation Interceptor restart when work preserving NM is enabled
> ---------------------------------------------------------------------
>
>                 Key: YARN-6704
>                 URL: https://issues.apache.org/jira/browse/YARN-6704
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Botong Huang
>            Assignee: Botong Huang
>         Attachments: YARN-6704-YARN-2915.v1.patch, YARN-6704-YARN-2915.v2.patch, YARN-6704.v3.patch,
YARN-6704.v4.patch, YARN-6704.v5.patch, YARN-6704.v6.patch
>
>
> YARN-1336 added the ability to restart NM without loosing any running containers. {{AMRMProxy}}
restart is added in YARN-6127. In a Federated YARN environment, there's additional state in
the {{FederationInterceptor}} to allow for spanning across multiple sub-clusters, so we need
to enhance {{FederationInterceptor}} to support work-preserving restart.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message