hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richa Khandelwal <richa...@gmail.com>
Subject Numbers of mappers and reducers
Date Tue, 17 Mar 2009 16:18:37 GMT
I was going through FAQs on Hadoop to optimize the performance of
map/reduce. There is a suggestion to set the number of reducers to a prime
number closest to the number of nodes and number of mappers a prime number
closest to several times the number of nodes in the cluster.
What performance advantages do these numbers give? Obviously doing so
improved the performance of my map reduce jobs considerably. Interested to
know the principles behind it.

Richa Khandelwal

University Of California,
Santa Cruz.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message