hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahesh Balija <balijamahesh....@gmail.com>
Subject Re: mappers-node relationship
Date Fri, 25 Jan 2013 09:18:00 GMT
Mappers and Reducers will run in Task instances mapper/reducer instances
also called as mapper/reducer slots.
Each node can have multiple slots (I mean multiple mapper instances, each
run in a child JVM). And this is configurable with properties like
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum.
Also they run in parallel.

Best,
Mahesh Balija,
CalsoftLabs.



On Fri, Jan 25, 2013 at 1:16 PM, jamal sasha <jamalshasha@gmail.com> wrote:

> Hi.
>   A very very lame question.
> Does numbers of mapper depends on the number of nodes I have?
> How I imagine map-reduce is this.
> For example in word count example
> I have bunch of slave nodes.
> The documents are distributed across these slave nodes.
> Now depending on how big the data is, it will spread across the slave
> nodes.. and that is how my number of mappers are decided.
> I am sure, this is wrong understanding. As in pseudo-distributed node, i
> can see multiple mappers.
> So question is.. how does a single node machine runs multiple mappers? is
> it run in parallel or sequentially??
> Any resources to learn these
> Thanks
>

Mime
View raw message