hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chih-Hsien Wu <chjaso...@gmail.com>
Subject Re: Relationship between heap sizes and mapred.child.java.opt configuration
Date Mon, 25 Nov 2013 17:12:18 GMT
Thanks for the reply. So what is the purpose of heap sizes for tasktrackers
and datanodes then? In other words, if I want to speed up the map/reducing
cycle, can I just minimize the heap size and maximize the
"mapred.child.java.opts?" or will the minimizing heap sizes causing out of
memory exception?


On Mon, Nov 25, 2013 at 10:02 AM, Kai Voigt <k@123.org> wrote:

> mapred.child.java.opts are referring to the settings for the JVMs spawned
> by the TaskTracker. This JVMs will actually run the tasks (mappers and
> reducers)
>
> The heap sizes for TaskTrackers and DataNodes are unrelated to those. They
> run in their own JVMs each.
>
> Kai
>
> Am 25.11.2013 um 15:52 schrieb Chih-Hsien Wu <chjasonwu@gmail.com>:
>
> I'm learning about Hadoop configuration. What is the connection between
> the datanode/ tasktracker heap sizes and the "mapre.child.java.opts"?  Does
> one have to be exceeded to another?
>
>
> ------------------------------
> *Kai Voigt* Am Germaniahafen 1 k@123.org
> 24143 Kiel +49 160 96683050
> Germany @KaiVoigt
>
>

Mime
View raw message