hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7369) Erasure coding: distribute block recovery work to DataNode
Date Thu, 12 Feb 2015 19:42:13 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhe Zhang updated HDFS-7369:
    Attachment: HDFS-7396-000.patch

A very simple patch for step 1 in the JIRA description. Ultimately we want to accurately assess
the _risk level_ of a block group when some of its EC blocks are lost. This patch is the first
order approximation:
# A block group with _m_ data blocks and _k_ parity blocks is equivalent to a contiguous block
with replication factor _k+1_, because both can tolerate _k_ failures
# If _n_ blocks are healthy among the _m+k_ blocks, it's equivalent to having _n-m+1_ healthy
replicas, calculated from _(k+1) - ((m+k) - n)_, or _replicationFactor - numLostReplicas_.

> Erasure coding: distribute block recovery work to DataNode
> ----------------------------------------------------------
>                 Key: HDFS-7369
>                 URL: https://issues.apache.org/jira/browse/HDFS-7369
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7396-000.patch
> This JIRA updates NameNode to handle background / offline recovery of erasure coded blocks.
It includes 2 parts:
> # Extend {{UnderReplicatedBlocks}} to recognize EC blocks and insert them to appropriate
priority levels. 
> # Update {{ReplicationMonitor}} to distinguish block codec tasks and send a new DataNode

This message was sent by Atlassian JIRA

View raw message