hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bing Jiang <jiangbinglo...@gmail.com>
Subject Re: Haoop: Container is running beyond physical memory limits
Date Fri, 03 Jan 2014 07:58:21 GMT
If memory of node manager is enough, please boost up the  physical memory,
and heap size of reduce task.
-Dmapreduce.reduce.memory.mb=6144 -Dmapreduce.reduce.java.opts="-Xmx5120M"


2014/1/3 Baron Tsai <tsaizehua@gmail.com>

> Container [pid=8689,containerID=container_1388722003823_0001_01_000062] is
> running beyond physical memory limits. Current usage: 4.1 GB of 4 GB
> physical memory used; 4.9 GB of 8.4 GB virtual memory used. Killing
> container.
> Dump of the process-tree for container_1388722003823_0001_01_000062 :
> |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
>  |- 8698 8689 8689 8689 (java) 11526 1047 5116669952 1086914
> /home/work/jdk//bin/java -Djava.net.preferIPv4Stack=true
> -Dhadoop.metrics.log.level=WARN -Xmx4096M
> -Djava.io.tmpdir=/home/data/hadoop_tmp/nm-local-dir/usercache/work/appcache/application_1388722003823_0001/container_1388722003823_0001_01_000062/tmp
> -Dlog4j.configuration=container-log4j.properties
> -Dyarn.app.container.log.dir=/home/data/hadoop/logs/userlogs/application_1388722003823_0001/container_1388722003823_0001_01_000062
> -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
> org.apache.hadoop.mapred.YarnChild 10.209.67.42 57097
> attempt_1388722003823_0001_r_000000_0 62
>  |- 8689 31450 8689 8689 (bash) 0 0 110751744 310 /bin/bash -c
> /home/work/jdk//bin/java -Djava.net.preferIPv4Stack=true
> -Dhadoop.metrics.log.level=WARN  -Xmx4096M
> -Djava.io.tmpdir=/home/data/hadoop_tmp/nm-local-dir/usercache/work/appcache/application_1388722003823_0001/container_1388722003823_0001_01_000062/tmp
> -Dlog4j.configuration=container-log4j.properties
> -Dyarn.app.container.log.dir=/home/data/hadoop/logs/userlogs/application_1388722003823_0001/container_1388722003823_0001_01_000062
> -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
> org.apache.hadoop.mapred.YarnChild 10.209.67.42 57097
> attempt_1388722003823_0001_r_000000_0 62
> 1>/home/data/hadoop/logs/userlogs/application_1388722003823_0001/container_1388722003823_0001_01_000062/stdout
> 2>/home/data/hadoop/logs/userlogs/application_1388722003823_0001/container_1388722003823_0001_01_000062/stderr
>
> I tried some configurations, but still failed, who can do me a favor, thxs!
>



-- 
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

Mime
View raw message