hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Best way to limit the number of concurrent tasks per job on hadoop 0.20.2
Date Tue, 25 Jan 2011 15:20:19 GMT
Capacity Scheduler (or a version of it) does ship with the 0.20
release of Hadoop and is usable. It can be used to assign queues with
a limited capacity for each, which your jobs must appropriately submit
to if you want them to utilize only the assigned fraction of your
cluster for its processing.

On Tue, Jan 25, 2011 at 5:19 PM, Renaud Delbru <renaud.delbru@deri.org> wrote:
> Hi,
>
> we would like to limit the number of maximum tasks per job on our hadoop
> 0.20.2 cluster.
> Is the Capacity Scheduler [1] will allow to do this ? Is it correctly
> working on hadoop 0.20.2 (I remember a  few months ago, we were looking at
> it, but it seemed incompatible with hadoop 0.20.2).
>
> [1] http://hadoop.apache.org/common/docs/r0.20.2/capacity_scheduler.html
>
> Regards,
> --
> Renaud Delbru
>



-- 
Harsh J
www.harshj.com

Mime
View raw message