hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Dasgupta <gdsay...@gmail.com>
Subject Need info on mapred.child.java.opts, mapred.map.child.java.opts and mapred.reduce.child.java.opts
Date Fri, 08 Mar 2013 10:11:19 GMT

While I was reading about the important Hadoop configuration properties, I
came across a state of doubt regarding the Java heap space properties for
the child tasks. According to my understanding, *mapred.child.java.opts* is
the overall heap size allocated to any task (map or reduce). Then when we
are setting *mapred.map.child.java.opts* and
*mapred.reduce.child.java.opts*separately, are they overriding the

For example, if I have the following configuration:
*mapred.child.java.opts = -Xmx1g

mapred.map.child.java.opts = -Xmx2g

mapred.reduce.child.java.opts = -Xmx512m

*Then how exactly the memory allocation is getting distributed between map
and reduce? My mapper gets more than the overall heap space as specified or
it is restricted to 1g?
Can some one help me understand this concept? Also, what are the other heap
space related properties which we can use with the above and how?


View raw message