hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kawa <kawa.a...@gmail.com>
Subject Re: issue about set the memory for each container
Date Thu, 28 Nov 2013 21:26:20 GMT
For MapReduce jobs, you should use you configuration options in
mapred-site.xml or Configuration object in your Driver class.

mapreduce.map.memory.mb1536Larger resource limit for maps.
mapreduce.map.java.opts-Xmx1024MLarger heap-size for child jvms of maps.
mapreduce.reduce.memory.mb3072Larger resource limit for reduces.
mapreduce.reduce.java.opts-Xmx2560MLarger heap-size for child jvms of
reduces.

You can read more here
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuring_the_Hadoop_Daemons_in_Non-Secure_Mode
 and
http://grokbase.com/t/hadoop/hdfs-user/1324x8nt6d/how-mapred-map-child-java-opts-influence-the-container-resource-allocation


2013/11/26 ch huang <justlooks@gmail.com>

> hi,maillist:
>            i find each my container just use 200M heap space,how can i
> resize it?
>
> # ps -ef|grep -i yarnchild
> yarn     24333  8210 99 14:09 ?        00:00:05
> /usr/java/jdk1.7.0_25/bin/java -Djava.net.preferIPv4Stack=true
> -Dhadoop.metrics.log.level=WARN -Xmx200m
> -Djava.io.tmpdir=/data/mrlocal/1/yarn/local/usercache/hdfs/appcache/application_1385445543402_0006/container_1385445543402_0006_01_000017/tmp
> -Dlog4j.configuration=container-log4j.properties
> -Dyarn.app.mapreduce.container.log.dir=/data/mrlocal/2/yarn/logs/application_1385445543402_0006/container_1385445543402_0006_01_000017
> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
> org.apache.hadoop.mapred.YarnChild 192.168.10.224 59237
> attempt_1385445543402_0006_m_000015_0 17
>

Mime
View raw message