Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 12 Mar 2013 22:45:13 +0000 (UTC)
From: "Siddharth Seth (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12635218.1362434609964.429213.1363128313373@arcas>
In-Reply-To: <JIRA.12635218.1362434609964@arcas>
References: <JIRA.12635218.1362434609964@arcas>
Subject: [jira] [Commented] (YARN-449) MRAppMaster classpath not set
 properly for unit tests in downstream projects
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/YARN-449?page=3Dcom.atlassian.j=
ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D136005=
78#comment-13600578 ]=20

Siddharth Seth commented on YARN-449:
-------------------------------------

{code}
013-03-12 18:53:39,275 WARN  [Container Monitor] monitor.ContainersMonitorI=
mpl$MonitoringThread(444): Container [pid=3D8438,containerID=3Dcontainer_13=
63114400920_0001_01_000001] is running beyond virtual memory limits. Curren=
t usage: 217.9 MB of 2 GB physical memory used; 6.5 GB of 4.2 GB virtual me=
mory used. Killing container.
Dump of the process-tree for container_1363114400920_0001_01_000001 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TI=
ME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 8438 7023 8438 8438 (bash) 1 0 108650496 310 /bin/bash -c /usr/l=
ib/jvm/java-1.6.0-sun-1.6.0.37.x86_64/bin/java -Dlog4j.configuration=3Dcont=
ainer-log4j.properties -Dyarn.app.mapreduce.container.log.dir=3D/trunk/hbas=
e-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apach=
e.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_13631144=
00920_0001/container_1363114400920_0001_01_000001 -Dyarn.app.mapreduce.cont=
ainer.log.filesize=3D0 -Dhadoop.root.logger=3DINFO,CLA  -Xmx1024m org.apach=
e.hadoop.mapreduce.v2.app.MRAppMaster 1&gt;/trunk/hbase-server/target/org.a=
pache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniM=
RCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_=
1363114400920_0001_01_000001/stdout 2&gt;/trunk/hbase-server/target/org.apa=
che.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRC=
luster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_13=
63114400920_0001_01_000001/stderr
{code}

This is what caused TestRowCounter to fail in the linux env. Not sure why t=
he Vmem is going that high. The hadoop-1 default config likely disables thi=
s monitoring.

At this point there seem to be solutions for the original prolem the jira w=
as opened for, and this is really re-purposed to get HBase unit tests worki=
ng with Hadoop 2. Changing the title accordingly.
               =20
> MRAppMaster classpath not set properly for unit tests in downstream proje=
cts
> -------------------------------------------------------------------------=
---
>
>                 Key: YARN-449
>                 URL: https://issues.apache.org/jira/browse/YARN-449
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.0.3-alpha
>            Reporter: Siddharth Seth
>            Priority: Blocker
>         Attachments: 7904-v5.txt, hbase-7904-v3.txt, hbase-TestHFileOutpu=
tFormat-wip.txt, hbase-TestingUtility-wip.txt, minimr_randomdir-branch2.txt
>
>
> Post YARN-429, unit tests for HBase continue to fail since the classpath =
for the MRAppMaster is not being set correctly.
> Reverting YARN-129 may fix this, but I'm not sure that's the correct solu=
tion. My guess is, as Alexandro pointed out in YARN-129, maven classloader =
magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira