hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1040) De-link container life cycle from the process and add ability to execute multiple processes in the same long-lived container
Date Thu, 25 Feb 2016 00:58:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166462#comment-15166462
] 

Arun Suresh commented on YARN-1040:
-----------------------------------

So that we are on the same page, If we were to separate what needs to be in YARN vs what Slider
etc. should handle, id say :

*YARN*
* Container Upgrade primitive:
** provide AM with APIs (via NMClient) to upgrade the Container.
** API takes 1) new {{ContainerLaunchContext}} and 2) a policy viz. *In-place* (localize in
parallel v2, start v2, stop v1) or *New+rollback* (stop v1, localize v2, start v2) + (start
v1 if start v2 fails)  *or* list of primitive composable commands if the above policies doesn't
cover the use case.
** should negotiate Resource increase for in-place upgrade with RM prior to upgrade via YARN-1197
(or perhaps use OPPORTUNISTIC containers to locally negotiated at the NM for the resource
spike needed for upgrade, once YARN-2877 is ready)

*Slider / or something similar*
* Application upgrade primitive
** Upgrade Orchestration Policy: Allow applications deployed via slider to specify order in
which tasks/roles are upgraded (or started) 
** Allow applications to specify how containers of each role are upgraded
** Actually call the YARN container upgrade APIs (described above) to perform upgrade of each
container in the user specified order/policy

Makes sense ?


> De-link container life cycle from the process and add ability to execute multiple processes
in the same long-lived container
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-1040
>                 URL: https://issues.apache.org/jira/browse/YARN-1040
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>
> The AM should be able to exec >1 process in a container, rather than have the NM automatically
release the container when the single process exits.
> This would let an AM restart a process on the same container repeatedly, which for HBase
would offer locality on a restarted region server.
> We may also want the ability to exec multiple processes in parallel, so that something
could be run in the container while a long-lived process was already running. This can be
useful in monitoring and reconfiguring the long-lived process, as well as shutting it down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message