hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2261) YARN should have a way to run post-application cleanup
Date Tue, 08 Jul 2014 18:12:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055231#comment-14055231

Bikas Saha commented on YARN-2261:

+1 for having the control/responsibility in YARN
An alternative that may fit better with the RM model of launching AM's is to optionally, have
the RM run the AM in cleanup mode. This way the clean up logic can reside in the AM as it
does today and the RM does not need to learn any new tricks about launching anything other
that the AM. The existing launch context is used to launch the AM and the AM is told (via
env or via register) that its in cleanup mode. The AM can use its logic to do cleanup and
then successfully unregister with the RM. Until the unregister happens the RM can keep restarting
the AM in clean up mode for a max of N times (to handled unexpected failures). When an AM
is running in cleanup mode the it will not be allowed to make any allocated requests. This
can be handled via AMRMClient so that AM's that use AMRMClient dont need to do anything. The
ApplicationMasterService, of course, will need to handle this for non AMRMClient AM's.
By this method, minimal changes will be needed in the API and RM internals to enable this
feature in a compatible manner.

> YARN should have a way to run post-application cleanup
> ------------------------------------------------------
>                 Key: YARN-2261
>                 URL: https://issues.apache.org/jira/browse/YARN-2261
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: resourcemanager
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
> See MAPREDUCE-5956 for context. Specific options are at https://issues.apache.org/jira/browse/MAPREDUCE-5956?focusedCommentId=14054562&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14054562.

This message was sent by Atlassian JIRA

View raw message