hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Bhattacharjee <rahul.rec....@gmail.com>
Subject Hadoop schedulers!
Date Sat, 11 May 2013 15:01:28 GMT

I was going through the job schedulers of Hadoop and could not see any
major operational difference between the capacity scheduler and the fair
share scheduler apart from the fact that fair share scheduler supports
preemption and capacity scheduler doesn't.

Another thing is the former creates logical pools based on certain
attribute like username , user group etc and the later has a notion of job
queues. Can someone point me to any other major differences between these
two types of schedulers.

Another question in this regard is the capacity scheduler uses a FIFO
queue.So its still possible that a high priority long running job using all
the capacity allocated to the queue to block all the other jobs after it in
the queue.I think this is the expected behavior , but wanted to confirm.


View raw message