hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2114) re-commission of a decommissioned node does not delete excess replica
Date Thu, 14 Jul 2011 00:24:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064978#comment-13064978

Matt Foley commented on HDFS-2114:

bq. The first use of {isNodeDown} is necessary because {downnode} could be null in cases when
we are checking for "Recommission".

Yes, but in this case nodes[j].getName().equals(downnode) will return false without any problem.

bq. My understanding is that this assert ensures that the current blk has only ONE replica
that is in decommissioned state.

But it also seems to ensure that the only replica that is down is the LAST one in the array
of nodes for the block,
due to the way this block works:
      int firstDecomNodeIndex = -1;
      DatanodeInfo[] nodes = blk.getLocations();
      for (int j = 0; j < nodes.length; j++) {     // for each replica of blk
        if (nodes[j].isDecommissioned()) {
          if (firstDecomNodeIndex == -1) {
            firstDecomNodeIndex = j;
        assertEquals("Decom node is not at the end", firstDecomNodeIndex, -1);
Yet I don't see any code (eg in LocatedBlock.getLocations()) where downed nodes are moved
to the end of that list. Maybe somewhere else?

> * And in the same block, why is it important to condition it on 
> isNodeDown, since (!isNodeDown) implies there shouldn't be any 
> decommissioned nodes? So the second use of isNodeDown also seems 
> unnecessary.
Because, we don't care for this in cases where we there is no node that is down.
Right, but in this case the assertEquals("Decom node is not at the end", firstDecomNodeIndex,
-1) will always pass, because no nodes were decommissioned.

In summary, the use of the "isNodeDown" flag is only serving to delineate a human-interesting
state, but has no impact on the outcome of the code.  I would remove it, since the checkFile()
method works correctly without this flag when downnode == null or downnode != null.

> re-commission of a decommissioned node does not delete excess replica
> ---------------------------------------------------------------------
>                 Key: HDFS-2114
>                 URL: https://issues.apache.org/jira/browse/HDFS-2114
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: John George
>            Assignee: John George
>         Attachments: HDFS-2114-2.patch, HDFS-2114-3.patch, HDFS-2114.patch
> If a decommissioned node is removed from the decommissioned list, namenode does not delete
the excess replicas it created while the node was decommissioned.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message