hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: Running YARN in-process Application Master?
Date Tue, 29 Apr 2014 10:38:50 GMT
Yes there is. You can provide your own implementation of
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor and
configure it as  'yarn.nodemanager.container-executor.class' property.
There you can bypass Shell and create your own way of invoking processes.
Obviously it only makes sense for testing with mini-cluster during the
development and I think that's what you meant.
I've been playing with it lately, so feel free to take a look -
At least you'll get some ideas out of it.


On Mon, Apr 28, 2014 at 3:57 PM, bo yang <bobyangbo@gmail.com> wrote:

> Hi All,
> I just joined this group, and not sure whether this question was discussed
> before.
> Is it possible to run Application Master within the same process as Node
> Manager? If not, any plan to support it future?
> I am asking this because we might want to use YARN as a job dispatching
> system. We have our own heavy-weight service which is not practical to
> be launched as a new process for each new job. A possible thing we can do
> is to use YARN to launch a new dummy Application Master process which will
> dispatch the job to our service. But that new dummy process is a little
> waste. So if YARN supports in-process Application Master, that will be
> great.
> Thanks,
> Bo

View raw message