hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-15) All replicas of a block end up on only 1 rack
Date Mon, 24 Aug 2009 22:36:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-15?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747115#action_12747115

Jitendra Nath Pandey commented on HDFS-15:

In this patch, the blocks will stay in the list, but will not be scheduled for replication
because no new rack would be found to allocate sufficient racks.
  Suggestion from Hairong: 
               If user doesn't specify a topology script for rack determination, we can ignore
the check for enough racks.
  We can implement it by checking for config variable SCRIPT_FILENAME_KEY in blockHasEnoughRacks
function. If this config key returns null blockHasEnoughRacks will return true, which will
effectively eliminate the check for enough racks.

> All replicas of a block end up on only 1 rack
> ---------------------------------------------
>                 Key: HDFS-15
>                 URL: https://issues.apache.org/jira/browse/HDFS-15
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Hairong Kuang
>            Assignee: Jitendra Nath Pandey
>            Priority: Critical
>         Attachments: HDFS-15.patch, HDFS-15.patch.2
> HDFS replicas placement strategy guarantees that the replicas of a block exist on at
least two racks when its replication factor is greater than one. But fsck still reports that
the replicas of some blocks  end up on one rack.
> The cause of the problem is that decommission and corruption handling only check the
block's replication factor but not the rack requirement. When an over-replicated block loses
a replica due to decomission, corruption, or heartbeat lost, namenode does not take any action
to guarantee that remaining replicas are on different racks.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message