hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: number of reducers
Date Wed, 21 Nov 2012 04:08:34 GMT
Hey Jamal,

I'd recommend first going over the whole tutorial to get a good grip
on how Hadoop MR is designed to work:
http://hadoop.apache.org/docs/stable/mapred_tutorial.html

On Wed, Nov 21, 2012 at 1:08 AM, jamal sasha <jamalshasha@gmail.com> wrote:
>
>
> Hi,
>
>   I wrote a simple map reduce job in hadoop streaming.
>
>
>
> I am wondering if I am doing something wrong ..
>
> While number of mappers are projected to be around 1700.. reducers.. just 1?
>
> It’s couple of TB’s worth of data.
>
> What can I do to address this.
>
> Basically mapper looks like this
>
>
>
> For line in sys.stdin:
>
>     Print line
>
>
>
> Reducer
>
> For line in sys.stdin:
>
>     New_line = process_line(line)
>
>     Print new_line
>
>
>
>
>
> Thanks
>
>



-- 
Harsh J

Mime
View raw message