hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7344) [umbrella] Erasure Coding worker and support in DataNode
Date Thu, 19 Jan 2017 07:22:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829448#comment-15829448
] 

Zhe Zhang commented on HDFS-7344:
---------------------------------

Thanks for taking care of this [~andrew.wang]. I think HDFS-8014 and HDFS-8015 can be resolved
since HDFS-9719 (and possibly earlier JIRAs) has already introduced striped block reader and
writer. If we don't have one already, we can create a JIRA to track the consolidation of client
and DN -side block reader/writer code.

[~drankye] probably has more insights into HDFS-8019 and HDFS-8020. Looks to me they can be
closed as well.

> [umbrella] Erasure Coding worker and support in DataNode
> --------------------------------------------------------
>
>                 Key: HDFS-7344
>                 URL: https://issues.apache.org/jira/browse/HDFS-7344
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Kai Zheng
>            Assignee: Li Bo
>         Attachments: ECWorker-design-v2.pdf, hdfs-ec-datanode.0108.zip, hdfs-ec-datanode.0108.zip,
HDFS ECWorker Design.pdf
>
>
> According to HDFS-7285 and the design, this handles DataNode side extension and related
support for Erasure Coding. More specifically, it implements {{ECWorker}}, which reconstructs
lost blocks (in striping layout).
> It generally needs to restore BlockGroup and schema information from coding commands
from NameNode or other entities, and construct specific coding work to execute. The required
block reader, writer, either local or remote, encoder and decoder, will be implemented separately
as sub-tasks. 
> This JIRA will track all the linked sub-tasks, and is responsible for general discussions
and integration for ECWorker. It won't resolve until all the related tasks are done.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message