hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10968) BlockManager#isInNewRack should consider decommissioning nodes
Date Sat, 08 Oct 2016 06:04:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557318#comment-15557318

Hudson commented on HDFS-10968:

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10573 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10573/])
HDFS-10968. BlockManager#isInNewRack should consider decommissioning (jing9: rev 4d106213c0f4835b723c9a50bd8080a9017122d7)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReconstructStripedBlocksWithRackAwareness.java

> BlockManager#isInNewRack should consider decommissioning nodes
> --------------------------------------------------------------
>                 Key: HDFS-10968
>                 URL: https://issues.apache.org/jira/browse/HDFS-10968
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding, namenode
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>             Fix For: 3.0.0-alpha2
>         Attachments: HDFS-10968.000.patch
> For an EC block, it is possible we have enough internal blocks but without enough racks.
The current reconstruction code calls {{BlockManager#isInNewRack}} to check if the target
node can increase the total rack number for the case, which compares the target node's rack
with source node racks:
> {code}
>     for (DatanodeDescriptor src : srcs) {
>       if (src.getNetworkLocation().equals(target.getNetworkLocation())) {
>         return false;
>       }
>     }
> {code}
> However here the {{srcs}} may include a decommissioning node, in which case we should
allow the target node to be in the same rack with it.
> For e.g., suppose we have 11 nodes: h1 ~ h11, which are located in racks r1, r1, r2,
r2, r3, r3, r4, r4, r5, r5, r6, respectively. In case that an EC block has 9 live internal
blocks on (h1~h8 + h11), and one internal block on h9 which is to be decommissioned. The current
code will not choose h10 for reconstruction because isInNewRack thinks h10 is on the same
rack with h9.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message