hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7553) fix the TestDFSUpgradeWithHA due to BindException
Date Thu, 10 Dec 2015 18:37:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051427#comment-15051427

Xiao Chen commented on HDFS-7553:

Not sure if [~xieliang007] is still working on this, but I want to self-promote patch 2 with
a way to reproduce. Attached is the reproduce patch which includes patch 2, that fails with
the bind exception when restarting NN. Uncommenting the {{httpServer.join();}} at {{NameNode.java#932}}
makes the test pass. The test case I ran was {{testNfsUpgrade}}.

In the repro patch, the exception is thrown from {{HttpServer2.openListeners}}, instead of
the inner jetty + net classes. But IMHO this already proves that we need a {{join}} to have
a successful immediate NN restart. (Arguably though, with this the NN shutdown will be slower,
and potentially hang if somehow the webserver hangs in its shutdown. I think this would be
the webserver's bug, and could be enhanced in a separate JIRA. E.g. add a timeout to the NN

Thanks [~wheat9] for helping understand the problem, and [~cnauroth] for the initial thought
- turns out to be exactly the cause! Please feel free try out the repro patch and review patch

> fix the TestDFSUpgradeWithHA due to BindException
> -------------------------------------------------
>                 Key: HDFS-7553
>                 URL: https://issues.apache.org/jira/browse/HDFS-7553
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: test
>    Affects Versions: 2.7.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-7553-001.txt, HDFS-7553.002.patch, HDFS-7553.repro.patch
> see https://builds.apache.org/job/PreCommit-HDFS-Build/9092//testReport/org.apache.hadoop.hdfs.server.namenode.ha/TestDFSUpgradeWithHA/testNfsUpgrade/
> Error Message
> Port in use: localhost:57896
> Stacktrace
> java.net.BindException: Port in use: localhost:57896
> 	at sun.nio.ch.Net.bind0(Native Method)
> 	at sun.nio.ch.Net.bind(Net.java:444)
> 	at sun.nio.ch.Net.bind(Net.java:436)
> 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> 	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
> 	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:868)
> 	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:809)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:704)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:591)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:763)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:747)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1443)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1815)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1796)
> 	at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testNfsUpgrade(TestDFSUpgradeWithHA.java:285)

This message was sent by Atlassian JIRA

View raw message