hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3685) Unbalanced replication target
Date Wed, 02 Jul 2008 22:14:45 GMT
Unbalanced replication target 

                 Key: HADOOP-3685
                 URL: https://issues.apache.org/jira/browse/HADOOP-3685
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.17.0
            Reporter: Koji Noguchi
            Priority: Critical

In HADOOP-3633, namenode was assigning some datanodes to receive  hundreds of blocks in a
short period which caused datanodes to go out of memroy(threads).
Most of them were from remote rack.

Looking at the code, 

    166           chooseLocalRack(results.get(1), excludedNodes, blocksize,
    167                           maxNodesPerRack, results);

was sometimes not choosing the local rack of the writer(source).  

As a result, when a datanode goes down, other datanodes on the same rack were getting large
number of blocks from remote racks.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message