Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 11 Mar 2015 23:10:39 +0000 (UTC)
From: "Jing Zhao (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12753323.1415258848000.69461.1426115439102@Atlassian.JIRA>
In-Reply-To: <JIRA.12753323.1415258848000@Atlassian.JIRA>
References: <JIRA.12753323.1415258848000@Atlassian.JIRA>
 <JIRA.12753323.1415258848654@arcas>
Subject: [jira] [Commented] (HDFS-7369) Erasure coding: distribute block
 recovery work to DataNode
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357774#comment-14357774 ] 

Jing Zhao commented on HDFS-7369:
---------------------------------

Thanks for working on this, Zhe! The patch looks good to me. Some comments and thoughts:
# For a striped block, I think it will be better to use BlockInfo(Striped), instead of its individual blocks, as the basic unit for recovery. E.g., suppose we lose 2 blocks for a 6+3 EC block. For recovery, I guess we want these two blocks are recovered in a single recovery work instead of 2.
# As you mentioned in HDFS-7912, {{BlockManager}} and {{ReplicationMonitor}} never see individual data/parity blocks currently. But it may be better to have a more strict type restriction in {{UnderReplicatedBlocks}}, {{ReplicationWork}}, {{ErasureCodingWork}}, and {{computeRecoveryWorkForBlocks}}'s parameter.
# Also because of #1, we may want to define a list of target DNs in {{BlockCodecInfo}}?

> Erasure coding: distribute block recovery work to DataNode
> ----------------------------------------------------------
>
>                 Key: HDFS-7369
>                 URL: https://issues.apache.org/jira/browse/HDFS-7369
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7369-000-part1.patch, HDFS-7369-000-part2.patch, HDFS-7369-001.patch
>
>
> This JIRA updates NameNode to handle background / offline recovery of erasure coded blocks. It includes 2 parts:
> # Extend {{UnderReplicatedBlocks}} to recognize EC blocks and insert them to appropriate priority levels. 
> # Update {{ReplicationMonitor}} to distinguish block codec tasks and send a new DataNode command.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)