hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Ravi <bharathra...@gmail.com>
Subject Re: Parallel mapred jobs in Yarn
Date Thu, 10 Nov 2011 03:38:29 GMT
Thanks a lot all!
I'll try configuring the capacity scheduler on yarn, and get back to you if
I run into any issues.

On 9 November 2011 10:41, Robert Evans <evans@yahoo-inc.com> wrote:

> The configuration options are somewhat different for yarn, then they are
> for MRV1.  You probably want to generate the documentation for yarn
>
> mvn site
>
> And then read through it about how to set up your cluster
>
>
> ./hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/target/site/index.html
>
> There is documentation about setting up the capacity scheduler too.  If
> you run into any issues then reply here and if the documentation needs to
> be cleaned up was can file a JIRA against the documentation.  The
> documentation is kind of new so it would be good to get some real feedback
> on it.
>
> --Bobby Evans
>
> On 11/9/11 3:07 AM, "Vinod Kumar Vavilapalli" <vinodkv@hortonworks.com>
> wrote:
>
> FairScheduler isn't ported yet to YARN. The default scheduler is there (
> which is FifoScheduler) and CapacityScheduler can be configured too.
>
> HTH,
> +Vinod
>
>
> On Wed, Nov 9, 2011 at 10:14 AM, Bharath Ravi <bharathravi1@gmail.com
> >wrote:
>
> > Thanks, Prashant!
> > I'll try Yarn out with the Fairscheduler.
> >
> > On 8 November 2011 01:01, Prashant Sharma <prashant.iiith@gmail.com>
> > wrote:
> >
> > > Yes! , you can do the same in yarn as well.
> > > -P
> > >
> > > On Tue, Nov 8, 2011 at 3:24 AM, Bharath Ravi <bharathravi1@gmail.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have a beginner's question:
> > > > How can I configure yarn to allow multiple parallel mapreduce jobs to
> > > run?
> > > > Currently, the execution is sequential: each submitted job waits for
> > the
> > > > previous to run.
> > > >
> > > > In MR1, this could be done by enabling the
> > > fairscheduler/capacityscheduler.
> > > > Is there a similar configuration in Yarn as well?
> > > >
> > > > Thanks a lot!
> > > > --
> > > > Bharath Ravi
> > > >
> > >
> >
> >
> >
> > --
> > Bharath Ravi
> >
>
>


-- 
Bharath Ravi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message