hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-659) Boost the priority of re-replicating blocks that are far from their replication target
Date Wed, 24 Jan 2007 00:48:53 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hairong Kuang updated HADOOP-659:
---------------------------------

        Fix Version/s: 0.11.0
    Affects Version/s:     (was: 0.7.2)
                       0.10.1
               Status: Patch Available  (was: Open)

> Boost the priority of re-replicating blocks that are far from their replication target
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-659
>                 URL: https://issues.apache.org/jira/browse/HADOOP-659
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.10.1
>            Reporter: Konstantin Shvachko
>         Assigned To: Hairong Kuang
>             Fix For: 0.11.0
>
>         Attachments: priBlockRep2.patch
>
>
> I see two types of replications that should be accelerated compared to all others.
> 1. Blocks that have only one remaining copy (but are required to have higher replication).
> 2. Blocks that have less than 1/3 of their replicas in place.
> The latter occurs when map/reduce sets replication of certain files to 10, and we want
> it happen fast to achieve better performance on the tasks.
> So I think we should distinguish two major groups of under-replicated blocks:
> first-priority (having only 1 copy or less than 1/3 of required replicas), and the rest.
> The name-node places first-priority blocks into the beginning of the neededReplication
> list, and the rest are placed at the end. That way the first-priority blocks will be
replicated
> first and then the others.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message