hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From panfei <cnwe...@gmail.com>
Subject Fwd: Container [pid=22885,containerID=container_1386156666044_0001_01_000013] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 332.5 GB of 8 GB virtual memory used. Killing container.
Date Wed, 04 Dec 2013 13:16:46 GMT
---------- Forwarded message ----------
From: panfei <cnweike@gmail.com>
Date: 2013/12/4
Subject: Container
[pid=22885,containerID=container_1386156666044_0001_01_000013] is running
beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical
memory used; 332.5 GB of 8 GB virtual memory used. Killing container.
To: CDH Users <cdh-user@cloudera.org>


Hi All:

We are using CDH4.5 Hadoop for our production, when submit some (not all)
jobs from hive, we get the following exception info , seems the physical
memory and virtual memory both not enough for the job to run:


Task with the most failures(4):
-----
Task ID:
  task_1386156666044_0001_m_000000

URL:

http://namenode-1:8088/taskdetails.jsp?jobid=job_1386156666044_0001&tipid=task_1386156666044_0001_m_000000
-----
Diagnostic Messages for this Task:
Container [pid=22885,containerID=container_1386156666044_0001_01_000013] is
running beyond physical memory limits. Current usage: 1.0 GB of 1 GB
physical memory used; 332.5 GB of 8 GB virtual memory used. Killing
container.
Dump of the process-tree for container_1386156666044_0001_01_000013 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 22885 22036 22885 22885 (java) 5414 108 356993519616 271953
/usr/java/default/bin/java -Djava.net.preferIPv4Stack=true
-Dhadoop.metrics.log.level=WARN -Xmx200m
-Djava.io.tmpdir=/data/yarn/local/usercache/hive/appcache/application_1386156666044_0001/container_1386156666044_0001_01_000013/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/var/log/hadoop-yarn/containers/application_1386156666044_0001/container_1386156666044_0001_01_000013
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
org.apache.hadoop.mapred.YarnChild 192.168.101.55 60841
attempt_1386156666044_0001_m_000000_3 13

following is some of our configuration:

  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>12288</value>
  </property>

  <property>
    <name>yarn.nodemanager.vmem-pmem-ratio</name>
    <value>8</value>
  </property>

  <property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
  </property>

  <property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>6</value>
  </property>

can you give me some advice? thanks a lot.
-- 
不学习,不知道



-- 
不学习,不知道

Mime
View raw message