hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15711) Fix branch-2 builds
Date Fri, 21 Sep 2018 17:44:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623935#comment-16623935

Konstantin Shvachko commented on HADOOP-15711:

[~aw] thanks a lot for your insight, this helps understanding the problem and differences
between the branches.

Looked at the PreCommit runs with the attached patch more closely. Definitely some cheating
there, as many tests were not executed at all (Just as [~aw] commented). This explains fast
running time.
 I also made Jenkins [execute branch-3.1 build|https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/12/console],
which completed in 12 hours.

I don't see any shortcut solutions to the branch-2 build problem at this point. We should
probably take the long path and understand the fixes that were done to unit tests on branch-3,
that were not backported to branch-2.

> Fix branch-2 builds
> -------------------
>                 Key: HADOOP-15711
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15711
>             Project: Hadoop Common
>          Issue Type: Task
>            Reporter: Jonathan Hung
>            Priority: Critical
>         Attachments: HADOOP-15711.001.branch-2.patch
> Branch-2 builds have been disabled for a while: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/
> A test run here causes hdfs tests to hang: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/4/
> Running hadoop-hdfs tests locally reveal some errors such as:{noformat}[ERROR] testComplexAppend2(org.apache.hadoop.hdfs.TestFileAppend2)
 Time elapsed: 0.059 s  <<< ERROR!
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:714)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1164)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1128)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1172)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:403)
>         at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:234)
>         at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1080)
>         at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:883)
>         at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:514)
>         at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
>         at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:489)
>         at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend2(TestFileAppend2.java:543)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){noformat}
> I was able to get more tests passing locally by increasing the max user process count
on my machine. But the error suggests that there's an issue in the tests themselves. Not sure
if the error seen locally is the same reason as why jenkins builds are failing, I wasn't able
to confirm based on the jenkins builds' lack of output.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message