hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: in YARN/MR2, can I still submit multiple jobs to one MR application master?
Date Tue, 28 Apr 2015 00:07:50 GMT
The MapReduce ApplicationMaster supports only one job. You can say that (YARN ResourceManager
+ a bunch of MR ApplicationMasters (one per job) = JobTracker).

Tez does have a notion of multiple DAGs per YARN app.

For your specific use-case, you can force that user to a queue and limit how much he/she can
access.

Thanks
+Vinod

On Apr 27, 2015, at 3:30 PM, Yang <teddyyyy123@gmail.com> wrote:

> conceptually, the MR application master is similar to the old job tracker.
> 
> if so, can I submit multiple jobs to the same MR application master?  it looks like an
odd use case, the context is that we have users generating lots of MR jobs, and he currently
has a little crude scheduler that periodically launches jobs to the RM by just "hadoop jar
..."
> 
> instead I was thinking to "carve out" a MR2 allocation in RM first, then periodically
submit to the "job tracker"/application master, so that all the jobs are localized to this
allocation.
> 
> 
> I was also thinking about using Tez instead of MR application master. Tez replaces MR2
application master, not on top of it, right?
> 
> Thanks
> Yang


Mime
View raw message