hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1040) De-link container life cycle from the process and add ability to execute multiple processes in the same long-lived container
Date Thu, 25 Feb 2016 19:14:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167674#comment-15167674

Arun Suresh commented on YARN-1040:

Thanks for clarifying [~bikassaha]

I propose we break this down into sub-jiras :
# New APIs specific to delinking container life-cycle from the process : This will include
4 of new APIs specified above (excluding localization) but will assume single process (and
this the startProcess does not need to return a processId)
# Add support for clubbing APIs into a single RPC
** Might have to think a bit about validating the order and multiplicity of the API calls
in each command (which I expect might be different for single process / multiple processes)

# Add support for localize API
# Add support for Multiple processes
** A processId will be returned for a startProcess. Might have to think thru this further.
for eg. how does this integrate with YARN-4725

For the purpose of Application Upgrades (for which this JIRA is marked as a sub-task of...
also why im calling it out specifically) : Add support for Container Upgrades
# Expose a canned NMCommand that has the list of APIs to upgrade based on some policy

If folks are fine with this, I will ahead and open JIRAs and link this issue to each of the
above JIRAs (Since I don't think I can create subtasks for this JIRA) so that we can start
work on the same..

> De-link container life cycle from the process and add ability to execute multiple processes
in the same long-lived container
> ----------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-1040
>                 URL: https://issues.apache.org/jira/browse/YARN-1040
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
> The AM should be able to exec >1 process in a container, rather than have the NM automatically
release the container when the single process exits.
> This would let an AM restart a process on the same container repeatedly, which for HBase
would offer locality on a restarted region server.
> We may also want the ability to exec multiple processes in parallel, so that something
could be run in the container while a long-lived process was already running. This can be
useful in monitoring and reconfiguring the long-lived process, as well as shutting it down.

This message was sent by Atlassian JIRA

View raw message