hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamal sasha <jamalsha...@gmail.com>
Subject number of reducers
Date Tue, 20 Nov 2012 19:38:54 GMT
Hi,

  I wrote a simple map reduce job in hadoop streaming.



I am wondering if I am doing something wrong ..

While number of mappers are projected to be around 1700.. reducers.. just 1?

It’s couple of TB’s worth of data.

What can I do to address this.

Basically mapper looks like this



For line in sys.stdin:

    Print line



Reducer

For line in sys.stdin:

    New_line = process_line(line)

    Print new_line





Thanks

Mime
View raw message