hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6616) bestNode shouldn't always return the first DataNode
Date Fri, 18 Jul 2014 18:40:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066713#comment-14066713
] 

Hudson commented on HDFS-6616:
------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #5916 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5916/])
HDFS-6616. Add exclude-datanodes feature to WebHDFS redirection so that it will not redirect
retries to the same datanode. Contributed by zhaoyunjiong (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611750)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/ExcludeDatanodesParam.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java


> bestNode shouldn't always return the first DataNode
> ---------------------------------------------------
>
>                 Key: HDFS-6616
>                 URL: https://issues.apache.org/jira/browse/HDFS-6616
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>            Reporter: zhaoyunjiong
>            Assignee: zhaoyunjiong
>            Priority: Minor
>             Fix For: 2.6.0
>
>         Attachments: HDFS-6616.1.patch, HDFS-6616.2.patch, HDFS-6616.3.patch, HDFS-6616.patch
>
>
> When we are doing distcp between clusters, job failed:
> 014-06-30 20:56:28,430 INFO org.apache.hadoop.tools.DistCp: FAIL part-r-00101.avro :
java.net.NoRouteToHostException: No route to host
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1491)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1485)
> 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139)
> 	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
> 	at org.apache.hadoop.hdfs.HftpFileSystem.open(HftpFileSystem.java:322)
> 	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> 	at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:419)
> 	at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:547)
> 	at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:314)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> The root reason is one of the DataNode can't access from outside, but inside cluster,
it's health.
> In NamenodeWebHdfsMethods.java:bestNode, it always return the first DataNode, so even
after the distcp retries, it still failed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message