hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adeel Qureshi <adeelmahm...@gmail.com>
Subject secondary sort - number of reducers
Date Thu, 29 Aug 2013 23:23:01 GMT
I have implemented secondary sort in my MR job and for some reason if i
dont specify the number of reducers it uses 1 which doesnt seems right
because im working with 800M+ records and one reducer slows things down
significantly. Is this some kind of limitation with the secondary sort that
it has to use a single reducer .. that kind of would defeat the purpose of
having a scalable solution such as secondary sort. I would appreciate any
help.

Thanks
Adeel

Mime
View raw message