hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6704) Add Federation Interceptor restart when work preserving NM is enabled
Date Sat, 15 Jul 2017 01:22:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088357#comment-16088357

Subru Krishnan commented on YARN-6704:

Thanks [~botong] for the patch. I looked and have a few questions.

What's the rationale behind refactoring of _createAndRegisterNewUAM_ to _launchUAM_? Additionally
why would a client launch a UAM by already providing the _UnmanagedAMIdentifier_? I don't
even see the _UnmanagedAMIdentifier_ being used anywhere.

I feel we shouldn't be persisting the running containers in the NM store:
* This explodes the complexity from O(apps) --> O(apps*containers).
* Moreover, the running containers are recoverable as they are part of the {{RegisterApplicationMasterResponse}}.

A nit: there seems to be multiple looping through Map data structures (even in tests) when
it looks like a direct lookup by key is possible.

> Add Federation Interceptor restart when work preserving NM is enabled
> ---------------------------------------------------------------------
>                 Key: YARN-6704
>                 URL: https://issues.apache.org/jira/browse/YARN-6704
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Botong Huang
>            Assignee: Botong Huang
>         Attachments: YARN-6704-YARN-2915.v1.patch, YARN-6704-YARN-2915.v2.patch
> YARN-1336 added the ability to restart NM without loosing any running containers. {{AMRMProxy}}
restart is added in YARN-6127. In a Federated YARN environment, there's additional state in
the {{FederationInterceptor}} to allow for spanning across multiple sub-clusters, so we need
to enhance {{FederationInterceptor}} to support work-preserving restart.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message