hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Ideas for dynamic change reducer task number ?
Date Mon, 23 Nov 2009 00:48:15 GMT

My concern is that it is just like hard code to use conf.setNumReduceTasks
on the configuration. It is not flexible, so my idea is that adding an
interface to change the reducer number dynamically according the different
size of input data set.

Jeff Zhang

On Sun, Nov 22, 2009 at 11:10 AM, Owen O'Malley <owen.omalley@gmail.com>wrote:

> I'd suggest trying to do conf.setNumReduceTasks on the configuration passed
> to the InputFormat in getSplits. It will probably just work.
> -- Owen

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message