hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8005) Erasure Coding: simplify striped block recovery work computation and add tests
Date Mon, 20 Apr 2015 21:56:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503759#comment-14503759
] 

Zhe Zhang commented on HDFS-8005:
---------------------------------

Good thoughts Walter. This is an important aspect we should take care of in the decomm process.
But with the current EC reconstruction logic, the selected source replicas are not re-replicated
either. They are used to reconstruct the lost replicas and then discarded. It would be cool
if we can save those to-be-decommissioned EC replicas and push them to new destinations as
well. Let's file an optimization JIRA under HDFS-8031?

> Erasure Coding: simplify striped block recovery work computation and add tests
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-8005
>                 URL: https://issues.apache.org/jira/browse/HDFS-8005
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>             Fix For: HDFS-7285
>
>         Attachments: HDFS-8005.000.patch, HDFS-8005.000.patch, HDFS-8005.001.patch
>
>
> HDFS-7369 adds the functionality to distribute recovery work of striped blocks to datanodes.
There are still some pending issues:
> # In {{BlockManager#chooseSourceNode}}, a node is added into {{healthyIndices}} without
checking if its block is live and healthy
> # The test {{TestRecoverStripedBlcoks#testMissingStripedBlock}} has not tested striped
blocks because the file is created before setting the storage policy
> # In {{computeRecoveryWorkForBlocks}}, instead of using {{BlockCollection#isStriped}},
we'd better use {{BlockInfo#isStriped}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message