Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 11 Jun 2013 14:08:33 +0000 (UTC)
From: "Eric Sirianni (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12652206.1370959700857.99007.1370959713334@arcas>
In-Reply-To: <JIRA.12652206.1370959700857@arcas>
References: <JIRA.12652206.1370959700857@arcas>
Subject: [jira] [Created] (HDFS-4898)
 BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly
 fallback to local rack
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Eric Sirianni created HDFS-4898:
-----------------------------------

             Summary: BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack
                 Key: HDFS-4898
                 URL: https://issues.apache.org/jira/browse/HDFS-4898
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: namenode
    Affects Versions: 2.0.4-alpha, 1.2.0
            Reporter: Eric Sirianni
            Priority: Minor


As currently implemented, {{BlockPlacementPolicyWithNodeGroup}} does not properly fallback to local rack when no nodes are available in remote racks, resulting in an improper {{NotEnoughReplicasException}}.

{code:title=BlockPlacementPolicyWithNodeGroup.java}
  @Override
  protected void chooseRemoteRack(int numOfReplicas,
      DatanodeDescriptor localMachine, HashMap<Node, Node> excludedNodes,
      long blocksize, int maxReplicasPerRack, List<DatanodeDescriptor> results,
      boolean avoidStaleNodes) throws NotEnoughReplicasException {
    int oldNumOfReplicas = results.size();
    // randomly choose one node from remote racks
    try {
      chooseRandom(
          numOfReplicas,
          "~" + NetworkTopology.getFirstHalf(localMachine.getNetworkLocation()),
          excludedNodes, blocksize, maxReplicasPerRack, results,
          avoidStaleNodes);
    } catch (NotEnoughReplicasException e) {
      chooseRandom(numOfReplicas - (results.size() - oldNumOfReplicas),
          localMachine.getNetworkLocation(), excludedNodes, blocksize,
          maxReplicasPerRack, results, avoidStaleNodes);
    }
  }
{code}

As currently coded the {{chooseRandom()}} call in the {{catch}} block will never succeed as the set of nodes within the passed in node path (e.g. {{/rack1/nodegroup1}}) is entirely contained within the set of excluded nodes (both are the set of nodes within the same nodegroup as the node chosen first replica).

The bug is that the fallback {{chooseRandom()}} call in the catch block should be passing in the _complement_ of the node path used in the initial {{chooseRandom()}} call in the try block (e.g. {{/rack1}})  - namely:
{code}
NetworkTopology.getFirstHalf(localMachine.getNetworkLocation())
{code}

This will yield the proper fallback behavior of choosing a random node from _within the same rack_, but still excluding those nodes _in the same nodegroup_

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira