hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Reix (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-11617) Several tests are not stable (on OpenJDK / x86_64 / RHEL 7.1)
Date Wed, 25 Feb 2015 15:54:18 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-11617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tony Reix updated HADOOP-11617:
-------------------------------
    Description: 
I'm now porting Hadoop 2.4.1 on PPC64 HW.
In order to have an idea about the stability of the tests, I've run several times all Hadoop
tests in a RHEL 7.1 / x86_64 / OpenJDK environment.
I've run it 12 times, for now, on the EXACT same environment.
Using some tool, I have extracted the list of tests that do not always produce the same results,
listed below, showing some test unstability.

I 'm now doing the same for version 2.6.0 on OpenJDK/Ubuntu/x86_64.

Report gives:
test-name N: T F E S |
where:
- N is the number of the test log file
- and:
T: Tests run
F: Failure
E: Error
S: Skipped

As an example (see below for full data):
hdfs.server.datanode.TestDeleteBlockPool                      0:    2   0   0   0 |
........
hdfs.server.datanode.TestDeleteBlockPool                      4:    2   0   1   0 |
........
hdfs.server.datanode.TestDeleteBlockPool                     11:    2   0   0   0 |
means that this test, out of 12 runs, had an error once, with 0 Failure and 0 Skipped test.


  was:
I'm now porting Hadoop 2.4.1 on PPC64 HW.
In order to have an idea about the stability of the tests, I've run several times all Hadoop
tests in a RHEL 7.1 / x86_64 / OpenJDK environment.
I've run it 12 times, for now, on the EXACT same environment.
Using some tool, I have extracted the list of tests that do not always produce the same results,
listed below, showing some test unstability.

I 'm now doing the same for version 2.6.0 on OpenJDK/Ubuntu/x86_64.

Report gives:
test-name N: T F E S |
where:
- N is the number of the test log file
- and:
T: Tests run
F: Failure
E: Error
S: Skipped

As an example (see below for full data):
hdfs.server.datanode.TestDeleteBlockPool                      0:    2   0   0   0 |
........
hdfs.server.datanode.TestDeleteBlockPool                      4:    2   0   1   0 |
........
hdfs.server.datanode.TestDeleteBlockPool                     11:    2   0   0   0 |
means that this test, out of 12 runs, had an error once, with 0 Failure and 0 Skipped test.


# ./BuildUnstable.sh -A mvn.Test.OpenJDK.res00 mvn.Test.OpenJDK.res00a mvn.Test.OpenJDK.res01a
mvn.Test.OpenJDK.res01 mvn.Test.OpenJDK.res02 mvn.Test.OpenJDK.res03 mvn.Test.OpenJDK.res04
mvn.Test.OpenJDK.res05 mvn.Test.OpenJDK.res06 mvn.Test.OpenJDK.res07 mvn.Test.OpenJDK.res08
mvn.Test.OpenJDK.res09
Files:
0: mvn.Test.OpenJDK.res00
1: mvn.Test.OpenJDK.res00a
2: mvn.Test.OpenJDK.res01a
3: mvn.Test.OpenJDK.res01
4: mvn.Test.OpenJDK.res02
5: mvn.Test.OpenJDK.res03
6: mvn.Test.OpenJDK.res04
7: mvn.Test.OpenJDK.res05
8: mvn.Test.OpenJDK.res06
9: mvn.Test.OpenJDK.res07
10: mvn.Test.OpenJDK.res08
11: mvn.Test.OpenJDK.res09

Unstable Tests:
------------------------------------------------------------        T   F   E   S |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  0:    3   0   2   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  1:    3   0   0   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  2:    3   0   1   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  3:    3   0   2   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  4:    3   0   1   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  5:    3   0   2   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  6:    3   0   1   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  7:    3   0   1   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  8:    3   0   0   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold  9:    3   0   1   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 10:    3   0   0   0 |
hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold 11:    3   0   1   0 |
------------------------------------------------------------        T   F   E   S |
mapreduce.v2.app.TestRMContainerAllocator                     0:                  |
mapreduce.v2.app.TestRMContainerAllocator                     1:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     2:                  |
mapreduce.v2.app.TestRMContainerAllocator                     3:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     4:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     5:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     6:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     7:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     8:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                     9:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                    10:   15   0   0   0 |
mapreduce.v2.app.TestRMContainerAllocator                    11:   15   0   0   0 |
------------------------------------------------------------        T   F   E   S |
hdfs.server.datanode.TestDeleteBlockPool                      0:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      1:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      2:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      3:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      4:    2   0   1   0 |
hdfs.server.datanode.TestDeleteBlockPool                      5:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      6:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      7:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      8:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                      9:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                     10:    2   0   0   0 |
hdfs.server.datanode.TestDeleteBlockPool                     11:    2   0   0   0 |
------------------------------------------------------------        T   F   E   S |
http.TestHttpServerLifecycle                                  0:    7   0   0   0 |
http.TestHttpServerLifecycle                                  1:    7   0   0   0 |
http.TestHttpServerLifecycle                                  2:                  |
http.TestHttpServerLifecycle                                  3:                  |
http.TestHttpServerLifecycle                                  4:                  |
http.TestHttpServerLifecycle                                  5:    7   0   0   0 |
http.TestHttpServerLifecycle                                  6:                  |
http.TestHttpServerLifecycle                                  7:                  |
http.TestHttpServerLifecycle                                  8:    7   0   0   0 |
http.TestHttpServerLifecycle                                  9:    7   0   0   0 |
http.TestHttpServerLifecycle                                 10:    7   0   0   0 |
http.TestHttpServerLifecycle                                 11:    7   0   0   0 |
------------------------------------------------------------        T   F   E   S |
conf.TestConfServlet                                          0: 2283 106  68  27 |
conf.TestConfServlet                                          1: 2283 106  68  27 |
conf.TestConfServlet                                          2: 2276 106  68  27 |
conf.TestConfServlet                                          3: 2276 110  68  27 |
conf.TestConfServlet                                          4: 2276 106  68  27 |
conf.TestConfServlet                                          5: 2283 109  68  27 |
conf.TestConfServlet                                          6: 2276 109  68  27 |
conf.TestConfServlet                                          7: 2276 106  68  27 |
conf.TestConfServlet                                          8: 2283 109  68  27 |
conf.TestConfServlet                                          9: 2283 110  68  27 |
conf.TestConfServlet                                         10: 2283 106  68  27 |
conf.TestConfServlet                                         11: 2283 106  68  27 |
------------------------------------------------------------        T   F   E   S |
mapreduce.v2.app.job.impl.TestTaskAttempt                     0:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     1:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     2:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     3:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     4:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     5:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     6:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     7:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     8:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                     9:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                    10:   13   0   0   0 |
mapreduce.v2.app.job.impl.TestTaskAttempt                    11:   13   0   2   0 |
------------------------------------------------------------        T   F   E   S |
mapred.TestLocalContainerLauncher                             0:  251   0   2   0 |
mapred.TestLocalContainerLauncher                             1:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             2:  251   0   2   0 |
mapred.TestLocalContainerLauncher                             3:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             4:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             5:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             6:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             7:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             8:  266   0   2   0 |
mapred.TestLocalContainerLauncher                             9:  266   0   2   0 |
mapred.TestLocalContainerLauncher                            10:  266   0   2   0 |
mapred.TestLocalContainerLauncher                            11:  266   0   4   0 |
------------------------------------------------------------        T   F   E   S |
fs.TestLocalDirAllocator                                      0:   30   6   0   0 |
fs.TestLocalDirAllocator                                      1:   30   6   0   0 |
fs.TestLocalDirAllocator                                      2:   30   6   0   0 |
fs.TestLocalDirAllocator                                      3:   30   9   0   0 |
fs.TestLocalDirAllocator                                      4:   30   6   0   0 |
fs.TestLocalDirAllocator                                      5:   30   9   0   0 |
fs.TestLocalDirAllocator                                      6:   30   9   0   0 |
fs.TestLocalDirAllocator                                      7:   30   6   0   0 |
fs.TestLocalDirAllocator                                      8:   30   9   0   0 |
fs.TestLocalDirAllocator                                      9:   30   9   0   0 |
fs.TestLocalDirAllocator                                     10:   30   6   0   0 |
fs.TestLocalDirAllocator                                     11:   30   6   0   0 |
------------------------------------------------------------        T   F   E   S |
metrics2.impl.TestMetricsSystemImpl                           0:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           1:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           2:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           3:    6   1   0   0 |
metrics2.impl.TestMetricsSystemImpl                           4:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           5:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           6:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           7:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           8:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                           9:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                          10:    6   0   0   0 |
metrics2.impl.TestMetricsSystemImpl                          11:    6   0   0   0 |
------------------------------------------------------------        T   F   E   S |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  0:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  1:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  2:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  3:   58   1   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  4:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  5:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  6:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  7:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  8:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler  9:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 10:   58   0   0   0 |
yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 11:   58   0   0   0 |
------------------------------------------------------------        T   F   E   S |
yarn.server.resourcemanager.TestClientRMTokens                0:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                1:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                2:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                3:  592   2   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                4:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                5:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                6:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                7:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                8:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens                9:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens               10:  592   1   6   1 |
yarn.server.resourcemanager.TestClientRMTokens               11:  592   1   6   1 |
------------------------------------------------------------        T   F   E   S |
ipc.TestRPC                                                   0:   18   1   7   0 |
ipc.TestRPC                                                   1:   18   1   7   0 |
ipc.TestRPC                                                   2:   18   1   7   0 |
ipc.TestRPC                                                   3:   18   1   7   0 |
ipc.TestRPC                                                   4:   18   1   7   0 |
ipc.TestRPC                                                   5:   18   1   7   0 |
ipc.TestRPC                                                   6:   18   1   7   0 |
ipc.TestRPC                                                   7:   18   1   7   0 |
ipc.TestRPC                                                   8:   18   1   7   0 |
ipc.TestRPC                                                   9:   18   2   7   0 |
ipc.TestRPC                                                  10:   18   1   7   0 |
ipc.TestRPC                                                  11:   18   1   7   0 |
------------------------------------------------------------        T   F   E   S |
net.TestHdfsNetworkTopologyWithNodeGroup                      0: 2654  12  15  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      1: 2654  12  13  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      2: 2654  12  14  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      3: 2654  12  15  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      4: 2654  12  15  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      5: 2654  12  15  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      6: 2654  12  14  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      7: 2654  12  14  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      8: 2654  12  13  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                      9: 2654  12  14  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                     10: 2654  12  13  15 |
net.TestHdfsNetworkTopologyWithNodeGroup                     11: 2654  12  14  15 |


> Several tests are not stable (on OpenJDK / x86_64 / RHEL 7.1)
> -------------------------------------------------------------
>
>                 Key: HADOOP-11617
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11617
>             Project: Hadoop Common
>          Issue Type: Bug
>         Environment: RHEL 7.1 / x86_64
>            Reporter: Tony Reix
>
> I'm now porting Hadoop 2.4.1 on PPC64 HW.
> In order to have an idea about the stability of the tests, I've run several times all
Hadoop tests in a RHEL 7.1 / x86_64 / OpenJDK environment.
> I've run it 12 times, for now, on the EXACT same environment.
> Using some tool, I have extracted the list of tests that do not always produce the same
results, listed below, showing some test unstability.
> I 'm now doing the same for version 2.6.0 on OpenJDK/Ubuntu/x86_64.
> Report gives:
> test-name N: T F E S |
> where:
> - N is the number of the test log file
> - and:
> T: Tests run
> F: Failure
> E: Error
> S: Skipped
> As an example (see below for full data):
> hdfs.server.datanode.TestDeleteBlockPool                      0:    2   0   0   0 |
> ........
> hdfs.server.datanode.TestDeleteBlockPool                      4:    2   0   1   0 |
> ........
> hdfs.server.datanode.TestDeleteBlockPool                     11:    2   0   0   0 |
> means that this test, out of 12 runs, had an error once, with 0 Failure and 0 Skipped
test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message