hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8326) mapreduce.TestTableInputFormatScan times out frequently
Date Mon, 15 Apr 2013 17:02:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631860#comment-13631860
] 

Nick Dimiduk commented on HBASE-8326:
-------------------------------------

It's not creating 12 separate MiniMR clusters. The clusters are managed in {{@BeforeClass}}
and {{@AfterClass}} annotations; there's a single cluster for the test. I don't know how JUnit
manages test progress reporting so as to kill on time-out. It seems to me that each test runs
its own scan and then reports success/failure. I would assume that counts as a kind of heart-beat
to indicate the test is still operating.

>From the logs on http://54.241.6.143/job/HBase-0.94-Hadoop-2/org.apache.hbase$hbase/63/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan/testScanEmptyToAPP/,
it looks like the container.MonitoringThread is terminating these tests:

{noformat}
2013-04-12 04:09:04,188 WARN  [Container Monitor] monitor.ContainersMonitorImpl(287): Process
tree for container: container_1365739591883_0006_01_000001 has processes older than 1 iteration
running over the configured limit. Limit=3382286592, current usage = 3625213952
2013-04-12 04:09:04,190 WARN  [Container Monitor] monitor.ContainersMonitorImpl$MonitoringThread(436):
Container [pid=24367,containerID=container_1365739591883_0006_01_000001] is running beyond
virtual memory limits. Current usage: 154.3mb of 1.5gb physical memory used; 3.4gb of 3.1gb
virtual memory used. Killing container.
Dump of the process-tree for container_1365739591883_0006_01_000001 :
	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES)
RSSMEM_USAGE(PAGES) FULL_CMD_LINE
	|- 24367 11719 24367 24367 (bash) 0 3 110800896 301 /bin/bash -c /usr/java/default/bin/java
-Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1365739591883_0006/container_1365739591883_0006_01_000001
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1>/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1365739591883_0006/container_1365739591883_0006_01_000001/stdout
2>/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1365739591883_0006/container_1365739591883_0006_01_000001/stderr
  
	|- 24398 24367 24367 24367 (java) 581 62 3514413056 39188 /usr/java/default/bin/java -Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1365739591883_0006/container_1365739591883_0006_01_000001
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
{noformat}
                
> mapreduce.TestTableInputFormatScan times out frequently
> -------------------------------------------------------
>
>                 Key: HBASE-8326
>                 URL: https://issues.apache.org/jira/browse/HBASE-8326
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce, test
>    Affects Versions: 0.98.0, 0.94.7, 0.95.1
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>
> bq. It looks like since this change org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
has not passed a single time in the EC2 builds (http://54.241.6.143/job/HBase-0.94/). Change
was introduced in Build #72, since then this tests times out (or doesn't finish)
> via [~lhofhansl] in [HBASE-8140 comment|https://issues.apache.org/jira/browse/HBASE-8140?focusedCommentId=13629101&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13629101].

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message