hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-449) MRAppMaster classpath not set properly for unit tests in downstream projects
Date Tue, 12 Mar 2013 22:45:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600578#comment-13600578
] 

Siddharth Seth commented on YARN-449:
-------------------------------------

{code}
013-03-12 18:53:39,275 WARN  [Container Monitor] monitor.ContainersMonitorImpl$MonitoringThread(444):
Container [pid=8438,containerID=container_1363114400920_0001_01_000001] is running beyond
virtual memory limits. Current usage: 217.9 MB of 2 GB physical memory used; 6.5 GB of 4.2
GB virtual memory used. Killing container.
Dump of the process-tree for container_1363114400920_0001_01_000001 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES)
RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 8438 7023 8438 8438 (bash) 1 0 108650496 310 /bin/bash -c /usr/lib/jvm/java-1.6.0-sun-1.6.0.37.x86_64/bin/java
-Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_000001
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA  -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1&gt;/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_000001/stdout
2&gt;/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_000001/stderr
{code}

This is what caused TestRowCounter to fail in the linux env. Not sure why the Vmem is going
that high. The hadoop-1 default config likely disables this monitoring.

At this point there seem to be solutions for the original prolem the jira was opened for,
and this is really re-purposed to get HBase unit tests working with Hadoop 2. Changing the
title accordingly.
                
> MRAppMaster classpath not set properly for unit tests in downstream projects
> ----------------------------------------------------------------------------
>
>                 Key: YARN-449
>                 URL: https://issues.apache.org/jira/browse/YARN-449
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.0.3-alpha
>            Reporter: Siddharth Seth
>            Priority: Blocker
>         Attachments: 7904-v5.txt, hbase-7904-v3.txt, hbase-TestHFileOutputFormat-wip.txt,
hbase-TestingUtility-wip.txt, minimr_randomdir-branch2.txt
>
>
> Post YARN-429, unit tests for HBase continue to fail since the classpath for the MRAppMaster
is not being set correctly.
> Reverting YARN-129 may fix this, but I'm not sure that's the correct solution. My guess
is, as Alexandro pointed out in YARN-129, maven classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message