hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5891) webhdfs should not try connecting the DN during redirection
Date Thu, 13 Feb 2014 11:15:43 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900233#comment-13900233
] 

Hudson commented on HDFS-5891:
------------------------------

SUCCESS: Integrated in Hadoop-Yarn-trunk #480 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/480/])
HDFS-5891. webhdfs should not try connecting the DN during redirection. Contributed by Haohui
Mai (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1567810)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestHttpsFileSystem.java


> webhdfs should not try connecting the DN during redirection
> -----------------------------------------------------------
>
>                 Key: HDFS-5891
>                 URL: https://issues.apache.org/jira/browse/HDFS-5891
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode, webhdfs
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>             Fix For: 2.4.0
>
>         Attachments: HDFS-5891.000.patch, HDFS-5891.001.patch
>
>
> When the webhdfs server in NN serves an {{OPEN}} request, the NN will eventually redirect
the request to a DN. The current implementation intends to choose the active DNs. The code
always connects to the DN in a deterministic order to see whether it is active during redirection.
> Although it reduces the chance of the client from connecting to a failed DN, this is
problematic because:
> # It has no guarantees that the client can connect to that DN even if the NN can connect
to it.
> # It requires an additional network round-trip for every {{OPEN}} / {{CREATE}} request.
> This jira proposes that the NN should choose the DN based on the information of the data
node manager.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message