hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Doubts Regarding Capacity Scheduler's Function
Date Tue, 24 Nov 2009 16:23:50 GMT

On Nov 24, 2009, at 6:23 AM, Abhilaash wrote:
> I have installed hadoop and configured capacity scheduler with
> three queues and run three jobs simultaneously,but i found that
> the jobs are scheduled only to the default queue and the jobs were
> not scheduled to the other queues configured by me.Is there any
> more configuration to be done for scheduling of jobs to the user
> defined queues?

Each job needs to have the mapred.job.queue.name set to the right  
queue for the CS. It defaults to "default" queue.

>        I have a basic doubt, whether Capacity scheduler schedules  
> the job
> as a whole(say wordcount) or schedules the tasks(maps and reduces
> of the job) to the available nodes in the cluster.

Every Map-Reduce scheduler: DefaultScheduler, CapacityScheduler,  
FairScheduler, schedules one task at a time.

>        It would be very helpful if someone helps me to clearly  
> understand
> the working of the capacity scheduler.

http://hadoop.apache.org/common/docs/r0.20.0/capacity_scheduler.html -  
hope this helps.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message