hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1300) deletion of excess replicas does not take into account 'rack-locality'
Date Tue, 05 Jun 2007 21:45:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501704

dhruba borthakur commented on HADOOP-1300:

I have a slightly different approach that uses the cluster map to delete excess replicas.
Should work correctly, but probably needs more thought.

1. If all nodes are at the same height, then
    -- if any two nodes have the same parent, select one of the
       node (based on free space) as a candidate for deletion.
    -- replace each node by its parent node
2. If all nodes are not at the same height, then
    -- select node with max height, repce it with its parent node
    -- go to step 1

> deletion of excess replicas does not take into account 'rack-locality'
> ----------------------------------------------------------------------
>                 Key: HADOOP-1300
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1300
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Koji Noguchi
>            Assignee: Hairong Kuang
>         Attachments: excessDel.patch
> One rack went down today, resulting in one missing block/file.
> Looking at the log, this block was originally over-replicated. 
> 3 replicas on one rack and 1 replica on another.
> Namenode decided to delete the latter, leaving 3 replicas on the same rack.
> It'll be nice if the deletion is also rack-aware.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message