Return-Path: X-Original-To: apmail-hadoop-common-dev-archive@www.apache.org Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B8AAF17693 for ; Fri, 20 Feb 2015 16:21:14 +0000 (UTC) Received: (qmail 34326 invoked by uid 500); 20 Feb 2015 16:21:12 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 34167 invoked by uid 500); 20 Feb 2015 16:21:11 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 33842 invoked by uid 99); 20 Feb 2015 16:21:11 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Feb 2015 16:21:11 +0000 Date: Fri, 20 Feb 2015 16:21:11 +0000 (UTC) From: "Tony Reix (JIRA)" To: common-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HADOOP-11617) Several tests are not stable (on OpenJDK / x86_64 / RHEL 7.1) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Tony Reix created HADOOP-11617: ---------------------------------- Summary: Several tests are not stable (on OpenJDK / x86_64 / RHEL 7.1) Key: HADOOP-11617 URL: https://issues.apache.org/jira/browse/HADOOP-11617 Project: Hadoop Common Issue Type: Bug Environment: RHEL 7.1 / x86_64 Reporter: Tony Reix I'm now porting Hadoop 2.4.1 on PPC64 HW. In order to have an idea about the stability of the tests, I've run several times all Hadoop tests in a RHEL 7.1 / x86_64 / OpenJDK environment. I've run it 12 times, for now, on the EXACT same environment. Using some tool, I have extracted the list of tests that do not always produce the same results, listed below, showing some test unstability. I 'm now doing the same for version 2.6.0 on OpenJDK/Ubuntu/x86_64. Report gives: test-name N: T F E S | where: - N is the number of the test log file - and: T: Tests run F: Failure E: Error S: Skipped As an example (see below for full data): hdfs.server.datanode.TestDeleteBlockPool 0: 2 0 0 0 | ........ hdfs.server.datanode.TestDeleteBlockPool 4: 2 0 1 0 | ........ hdfs.server.datanode.TestDeleteBlockPool 11: 2 0 0 0 | means that this test, out of 12 runs, had an error once, with 0 Failure and 0 Skipped test. # ./BuildUnstable.sh -A mvn.Test.OpenJDK.res00 mvn.Test.OpenJDK.res00a mvn.Test.OpenJDK.res01a mvn.Test.OpenJDK.res01 mvn.Test.OpenJDK.res02 mvn.Test.OpenJDK.res03 mvn.Test.OpenJDK.res04 mvn.Test.OpenJDK.res05 mvn.Test.OpenJDK.res06 mvn.Test.OpenJDK.res07 mvn.Test.OpenJDK.res08 mvn.Test.OpenJDK.res09 Files: 0: mvn.Test.OpenJDK.res00 1: mvn.Test.OpenJDK.res00a 2: mvn.Test.OpenJDK.res01a 3: mvn.Test.OpenJDK.res01 4: mvn.Test.OpenJDK.res02 5: mvn.Test.OpenJDK.res03 6: mvn.Test.OpenJDK.res04 7: mvn.Test.OpenJDK.res05 8: mvn.Test.OpenJDK.res06 9: mvn.Test.OpenJDK.res07 10: mvn.Test.OpenJDK.res08 11: mvn.Test.OpenJDK.res09 Unstable Tests: ------------------------------------------------------------ T F E S | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 0: 3 0 2 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 1: 3 0 0 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 2: 3 0 1 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 3: 3 0 2 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 4: 3 0 1 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 5: 3 0 2 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 6: 3 0 1 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 7: 3 0 1 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 8: 3 0 0 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 9: 3 0 1 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 10: 3 0 0 0 | hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 11: 3 0 1 0 | ------------------------------------------------------------ T F E S | mapreduce.v2.app.TestRMContainerAllocator 0: | mapreduce.v2.app.TestRMContainerAllocator 1: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 2: | mapreduce.v2.app.TestRMContainerAllocator 3: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 4: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 5: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 6: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 7: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 8: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 9: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 10: 15 0 0 0 | mapreduce.v2.app.TestRMContainerAllocator 11: 15 0 0 0 | ------------------------------------------------------------ T F E S | hdfs.server.datanode.TestDeleteBlockPool 0: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 1: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 2: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 3: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 4: 2 0 1 0 | hdfs.server.datanode.TestDeleteBlockPool 5: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 6: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 7: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 8: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 9: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 10: 2 0 0 0 | hdfs.server.datanode.TestDeleteBlockPool 11: 2 0 0 0 | ------------------------------------------------------------ T F E S | http.TestHttpServerLifecycle 0: 7 0 0 0 | http.TestHttpServerLifecycle 1: 7 0 0 0 | http.TestHttpServerLifecycle 2: | http.TestHttpServerLifecycle 3: | http.TestHttpServerLifecycle 4: | http.TestHttpServerLifecycle 5: 7 0 0 0 | http.TestHttpServerLifecycle 6: | http.TestHttpServerLifecycle 7: | http.TestHttpServerLifecycle 8: 7 0 0 0 | http.TestHttpServerLifecycle 9: 7 0 0 0 | http.TestHttpServerLifecycle 10: 7 0 0 0 | http.TestHttpServerLifecycle 11: 7 0 0 0 | ------------------------------------------------------------ T F E S | conf.TestConfServlet 0: 2283 106 68 27 | conf.TestConfServlet 1: 2283 106 68 27 | conf.TestConfServlet 2: 2276 106 68 27 | conf.TestConfServlet 3: 2276 110 68 27 | conf.TestConfServlet 4: 2276 106 68 27 | conf.TestConfServlet 5: 2283 109 68 27 | conf.TestConfServlet 6: 2276 109 68 27 | conf.TestConfServlet 7: 2276 106 68 27 | conf.TestConfServlet 8: 2283 109 68 27 | conf.TestConfServlet 9: 2283 110 68 27 | conf.TestConfServlet 10: 2283 106 68 27 | conf.TestConfServlet 11: 2283 106 68 27 | ------------------------------------------------------------ T F E S | mapreduce.v2.app.job.impl.TestTaskAttempt 0: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 1: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 2: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 3: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 4: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 5: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 6: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 7: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 8: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 9: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 10: 13 0 0 0 | mapreduce.v2.app.job.impl.TestTaskAttempt 11: 13 0 2 0 | ------------------------------------------------------------ T F E S | mapred.TestLocalContainerLauncher 0: 251 0 2 0 | mapred.TestLocalContainerLauncher 1: 266 0 2 0 | mapred.TestLocalContainerLauncher 2: 251 0 2 0 | mapred.TestLocalContainerLauncher 3: 266 0 2 0 | mapred.TestLocalContainerLauncher 4: 266 0 2 0 | mapred.TestLocalContainerLauncher 5: 266 0 2 0 | mapred.TestLocalContainerLauncher 6: 266 0 2 0 | mapred.TestLocalContainerLauncher 7: 266 0 2 0 | mapred.TestLocalContainerLauncher 8: 266 0 2 0 | mapred.TestLocalContainerLauncher 9: 266 0 2 0 | mapred.TestLocalContainerLauncher 10: 266 0 2 0 | mapred.TestLocalContainerLauncher 11: 266 0 4 0 | ------------------------------------------------------------ T F E S | fs.TestLocalDirAllocator 0: 30 6 0 0 | fs.TestLocalDirAllocator 1: 30 6 0 0 | fs.TestLocalDirAllocator 2: 30 6 0 0 | fs.TestLocalDirAllocator 3: 30 9 0 0 | fs.TestLocalDirAllocator 4: 30 6 0 0 | fs.TestLocalDirAllocator 5: 30 9 0 0 | fs.TestLocalDirAllocator 6: 30 9 0 0 | fs.TestLocalDirAllocator 7: 30 6 0 0 | fs.TestLocalDirAllocator 8: 30 9 0 0 | fs.TestLocalDirAllocator 9: 30 9 0 0 | fs.TestLocalDirAllocator 10: 30 6 0 0 | fs.TestLocalDirAllocator 11: 30 6 0 0 | ------------------------------------------------------------ T F E S | metrics2.impl.TestMetricsSystemImpl 0: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 1: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 2: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 3: 6 1 0 0 | metrics2.impl.TestMetricsSystemImpl 4: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 5: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 6: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 7: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 8: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 9: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 10: 6 0 0 0 | metrics2.impl.TestMetricsSystemImpl 11: 6 0 0 0 | ------------------------------------------------------------ T F E S | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 0: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 1: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 2: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 3: 58 1 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 4: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 5: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 6: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 7: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 8: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 9: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 10: 58 0 0 0 | yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 11: 58 0 0 0 | ------------------------------------------------------------ T F E S | yarn.server.resourcemanager.TestClientRMTokens 0: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 1: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 2: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 3: 592 2 6 1 | yarn.server.resourcemanager.TestClientRMTokens 4: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 5: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 6: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 7: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 8: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 9: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 10: 592 1 6 1 | yarn.server.resourcemanager.TestClientRMTokens 11: 592 1 6 1 | ------------------------------------------------------------ T F E S | ipc.TestRPC 0: 18 1 7 0 | ipc.TestRPC 1: 18 1 7 0 | ipc.TestRPC 2: 18 1 7 0 | ipc.TestRPC 3: 18 1 7 0 | ipc.TestRPC 4: 18 1 7 0 | ipc.TestRPC 5: 18 1 7 0 | ipc.TestRPC 6: 18 1 7 0 | ipc.TestRPC 7: 18 1 7 0 | ipc.TestRPC 8: 18 1 7 0 | ipc.TestRPC 9: 18 2 7 0 | ipc.TestRPC 10: 18 1 7 0 | ipc.TestRPC 11: 18 1 7 0 | ------------------------------------------------------------ T F E S | net.TestHdfsNetworkTopologyWithNodeGroup 0: 2654 12 15 15 | net.TestHdfsNetworkTopologyWithNodeGroup 1: 2654 12 13 15 | net.TestHdfsNetworkTopologyWithNodeGroup 2: 2654 12 14 15 | net.TestHdfsNetworkTopologyWithNodeGroup 3: 2654 12 15 15 | net.TestHdfsNetworkTopologyWithNodeGroup 4: 2654 12 15 15 | net.TestHdfsNetworkTopologyWithNodeGroup 5: 2654 12 15 15 | net.TestHdfsNetworkTopologyWithNodeGroup 6: 2654 12 14 15 | net.TestHdfsNetworkTopologyWithNodeGroup 7: 2654 12 14 15 | net.TestHdfsNetworkTopologyWithNodeGroup 8: 2654 12 13 15 | net.TestHdfsNetworkTopologyWithNodeGroup 9: 2654 12 14 15 | net.TestHdfsNetworkTopologyWithNodeGroup 10: 2654 12 13 15 | net.TestHdfsNetworkTopologyWithNodeGroup 11: 2654 12 14 15 | [root@soe01x hadoop-common-FromIBMSOE-20150210]# -- This message was sent by Atlassian JIRA (v6.3.4#6332)