hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "skyfly_99 (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7344) Erasure Coding worker and support in DataNode
Date Thu, 08 Jan 2015 11:30:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

skyfly_99 updated HDFS-7344:
    Attachment: hdfs-ec-datanode.0108.zip

Attachment is the datanode side code only considering non-stripping encoding and decoding.
After importing stripping, client side also needs to do encode/decode work which is similar
to datanode side process, so I am trying to implement them under one unified model. The code
just uploaded may have many changes in the future but the basic idea remains the same.

> Erasure Coding worker and support in DataNode
> ---------------------------------------------
>                 Key: HDFS-7344
>                 URL: https://issues.apache.org/jira/browse/HDFS-7344
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Kai Zheng
>            Assignee: Li Bo
>         Attachments: HDFS ECWorker Design.pdf, hdfs-ec-datanode.0108.zip
> According to HDFS-7285 and the design, this handles DataNode side extension and related
support for Erasure Coding, and implements ECWorker. It mainly covers the following aspects,
and separate tasks may be opened to handle each of them.
> * Process encoding work, calculating parity blocks as specified in block groups and codec
> * Process decoding work, recovering data blocks according to block groups and codec schema;
> * Handle client requests for passive recovery blocks data and serving data on demand
while reconstructing;
> * Write parity blocks according to storage policy.

This message was sent by Atlassian JIRA

View raw message