hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChengHuang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5593) [Umbrella] Add support for YARN Allocation composed of multiple containers/processes
Date Thu, 21 Dec 2017 09:22:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299743#comment-16299743
] 

ChengHuang edited comment on YARN-5593 at 12/21/17 9:21 AM:
------------------------------------------------------------

Since YARN is preparing to schedule service tasks, I think it's time for yarn to support multiple
containers/processes in one raw container. AFAIK, service tasks generally have several processes
in one container instance, for example, one process for the main process, one for monitoring,
one for log collection, etc. 
How about YARN provides some thing like POD in k8s?  
We can regard the raw YARN container as an allocation, and add SubContainer concept in ContainerLaunchContextProto.
Each SubContainer can run as process or as one docker container. 


was (Author: bellflowerih):
Since YARN is preparing to schedule service tasks, I think it's time for yarn to support multiple
containers/processes in one raw container. AFAIK, service tasks generally have several processes
in one container instance, for example, one process for the main process, one for monitoring,
one for log collection, etc. 
How about YARN provides some thing like POD in k8s?  

> [Umbrella] Add support for YARN Allocation composed of multiple containers/processes
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-5593
>                 URL: https://issues.apache.org/jira/browse/YARN-5593
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>
> Opening this to explicitly call out and track some of the ideas that were discussed in
YARN-1040. Specifically the concept of an {{Allocation}} which can be used by an AM to start
multiple {{Containers}} against as long as the sum of resources used by all containers {{fitsIn()}}
the Resources leased to the {{Allocation}}.
> This is especially useful for AMs that might want to target certain operations (like
upgrade / restart) on specific containers / processes within an Allocation without fear of
losing the allocation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message