hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: Running YARN in-process Application Master?
Date Tue, 29 Apr 2014 10:38:50 GMT
Yes there is. You can provide your own implementation of
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor and
configure it as  'yarn.nodemanager.container-executor.class' property.
There you can bypass Shell and create your own way of invoking processes.
Obviously it only makes sense for testing with mini-cluster during the
development and I think that's what you meant.
I've been playing with it lately, so feel free to take a look -
https://github.com/olegz/yaya/wiki/CoreFeatures#in-jvm-container-executor.
At least you'll get some ideas out of it.

Cheers
Oleg


On Mon, Apr 28, 2014 at 3:57 PM, bo yang <bobyangbo@gmail.com> wrote:

> Hi All,
>
> I just joined this group, and not sure whether this question was discussed
> before.
>
> Is it possible to run Application Master within the same process as Node
> Manager? If not, any plan to support it future?
>
> I am asking this because we might want to use YARN as a job dispatching
> system. We have our own heavy-weight service which is not practical to
> be launched as a new process for each new job. A possible thing we can do
> is to use YARN to launch a new dummy Application Master process which will
> dispatch the job to our service. But that new dummy process is a little
> waste. So if YARN supports in-process Application Master, that will be
> great.
>
> Thanks,
> Bo
>
>

Mime
View raw message