hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lin Ma <lin...@gmail.com>
Subject hadoop queue
Date Mon, 28 Jan 2013 03:02:28 GMT
Hi guys,

It is mentioned below for Hadoop queue (
http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/), for the
comments "FIFO scheduling with priorities is used", does it mean for
resources (mapper/reducer slots) allocation inside each queue, two factors
((1) job submit order, i.e. FIFO, and (2) individual job priorities are
also considered)? So it means even if inside one queue, the resource
allocation is not fair between jobs?

*However, within each queue, FIFO scheduling with priorities is used,
except for one aspect – you can place a limit on percent of running tasks
per user, so that users share a cluster equally. In other words, the
capacity scheduler tries to simulate a separate FIFO/priority cluster for
each user and each organization, rather than performing fair sharing
between all jobs. *


View raw message