hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@gmail.com>
Subject Re: Task scheduler
Date Fri, 14 May 2010 01:34:13 GMT

> i am experimenting with hadoop. wanted to ask that is the Task distribution
> policy by job tracker pluggable if yes where in the code tree is it defined.

Take a look at o.a.h.mapred.TaskScheduler. That's the abstract class
that needs to be extended to define a new scheduling policy. Also,
please do take a look at the existing schedulers that extend this
class. There are 3-4 implementations including the default scheduler,
capacity scheduler, fairshare scheduler and dynamic priority
scheduler. It may be worthwhile to see if your ideas match any of the
existing implementations to some degree and then consider enhancing
those as a first option.


View raw message