hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Majid Azimi <majid.merk...@gmail.com>
Subject Does mapred.local.dir is important factor in reducer side?
Date Mon, 31 Dec 2012 16:14:16 GMT
hi guys,

Hadoop the definitive guide says:
intermediate results on the mapper side is written to local disk at
mapred.local.dir location so if this location does not have enough space
the map will fail.

I want to know if this is true on the reducer side. Output of all mappers
will merge at reducer side. In which location this merge happens? If that
location does not have enough space does reducer fail? What is the solution
for MapReduce jobs if intermediat results for some keys is more than local
disk of reducer?

View raw message