hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Sonnenberg <steveis...@gmail.com>
Subject Re: All reducers are not being utilized
Date Thu, 02 Aug 2012 21:01:47 GMT
If I have 2 nodes, and 150 input files in a single 'input' directory to
search using the 'grep' example, isn't it reasonable that both nodes would
be involved?

Thanks

On Thu, Aug 2, 2012 at 3:31 PM, Bejoy Ks <bejoy.hadoop@gmail.com> wrote:

> Hi Saurab/Steve
>
> From my understanding the schedulers in hadoop consider only data
> locality(for map tasks) and availability of slots for scheduling tasks on
> various nodes. Say if you have a 3 TT nodes with 2 reducer slots each
> (assume all slots are free) . If we execute a map reduce job with  3 reduce
> tasks there is no gaurentee that one task will be scheduled on each node.
> It can be like 2 in one node and 1 in another.
>
> Regards
> Bejoy KS
>



-- 
Steve Sonnenberg

Mime
View raw message