hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11134) Fix bind exceptions in TestRenameWhileOpen and TestPendingInvalidateBlock
Date Wed, 16 Nov 2016 06:52:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669637#comment-15669637

Vinayakumar B commented on HDFS-11134:

bq. If this sentence is correct, we need to specify the same port. Hi Yiqun Lin and Brahma
Reddy Battula, what do you think? If this is true, I'm thinking it's better to retry the test
when BindException occurs.
I dont think for any test in {{TestRenameWhileOpen}} same namenode port is required. Because
all the streams and clients created before restart were unused after restart.
I have verified in my local machine that without specifying the name all tests passed. There
could be streamer threads running with different namenode addresses, which wont harm the tests.
So for safety, these streams could be aborted and old clients could be closed before creating
new client after restart.

For {{TestPendingInvalidateBlock}}, its not only {{cluster.restartDataNode(..);}} which tries
to restart the datanodes in same port, but {{cluster.restartNameNode(..);}} also restarts
the namenode in same port. In-fact {{cluster.restartNameNode(..);}} calls are more than {{cluster.restartDataNode(..);}}.
So I dont think changing only {{cluster.restartDataNode(..);}} will solve failures in {{TestPendingInvalidateBlock}}.

> Fix bind exceptions in TestRenameWhileOpen and TestPendingInvalidateBlock
> -------------------------------------------------------------------------
>                 Key: HDFS-11134
>                 URL: https://issues.apache.org/jira/browse/HDFS-11134
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Yiqun Lin
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11134.001.patch, HDFS-11134.002.patch
> I found two bind exceptions causing the unit tests fails in history Jenkins buildings.
One is test {{TestRenameWhileOpen}}, the other one is {{TestPendingInvalidateBlock}}.
> Here are the stack infos of {{TestRenameWhileOpen}}(I can't find the stack infos of {{TestPendingInvalidateBlock}}
now since the time of that happened is too early, but I'm sure it has failed due to bind exception.)
> {code}
> java.net.BindException: Problem binding to [localhost:42155] java.net.BindException:
Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
> 	at sun.nio.ch.Net.bind0(Native Method)
> 	at sun.nio.ch.Net.bind(Net.java:433)
> 	at sun.nio.ch.Net.bind(Net.java:425)
> 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> 	at org.apache.hadoop.ipc.Server.bind(Server.java:535)
> 	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:919)
> 	at org.apache.hadoop.ipc.Server.<init>(Server.java:2667)
> 	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:959)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:367)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:342)
> 	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:801)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:434)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:796)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:723)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:916)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1633)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1263)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1032)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:907)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:839)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
> 	at org.apache.hadoop.hdfs.TestRenameWhileOpen.testWhileOpenRenameToNonExistentDirectory(TestRenameWhileOpen.java:325)
> {code}
> Here specifying the namenode port is not necessary, is similar to HDFS-11129. And I have
run this test many times in my local, it all passed. The same we shuld do for test {{TestPendingInvalidateBlock}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message