hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "BoYang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2518) Support in-process container executor
Date Sun, 14 Sep 2014 05:20:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133089#comment-14133089
] 

BoYang commented on YARN-2518:
------------------------------

Yes, YARN-896 is a similar scenario. Thanks for bring in that. Is there anyone currently looking
at that?

By the way, can you provide more details about "launch a container and reuse it"? Let's say
the first job comes in and launches an application master container (process). Now the second
job comes in. YARN will still launch another application master container (process) for the
second one. How to make the second job reuse the previous application master container?

If you mean reuse the non-application-master containers, that might be possible. But for my
scenario, it needs to reuse the application master container. Is there anyway to do that?

> Support in-process container executor
> -------------------------------------
>
>                 Key: YARN-2518
>                 URL: https://issues.apache.org/jira/browse/YARN-2518
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager
>    Affects Versions: 2.5.0
>         Environment: Linux, Windows
>            Reporter: BoYang
>            Priority: Minor
>
> Node Manage always creates a new process for a new application. We have hit a scenario
where we want the node manager to execute the application inside its own process, so we get
fast response time. It would be nice if Node Manager or YARN can provide native support for
that.
> In general, the scenario is that we have a long running process which can accept requests
and process the requests inside its own process. Since YARN is good at scheduling jobs, we
want to use YARN to dispatch jobs (e.g. requests in JSON) to the long running process. In
that case, we do not want YARN container to spin up a new process for each request. Instead,
we want YARN container to send the request to the long running process for further processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message