hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7345) Local Reconstruction Codes (LRC)
Date Tue, 11 Nov 2014 21:23:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207092#comment-14207092

Andrew Wang commented on HDFS-7345:

I filed LEGAL-211, hopefully the legal team can provide some guidance on this matter.

> Local Reconstruction Codes (LRC)
> --------------------------------
>                 Key: HDFS-7345
>                 URL: https://issues.apache.org/jira/browse/HDFS-7345
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
> HDFS-7285 proposes to support Erasure Coding inside HDFS, supports multiple Erasure Coding
codecs via pluggable framework and implements Reed Solomon code by default. This is to support
a more advanced coding mechanism, Local Reconstruction Codes (LRC). As discussed in the paper
(https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf), LRC reduces the
number of erasure coding fragments that need to be read when reconstructing data fragments
that are offline, while still keeping the storage overhead low. The important benefits of
LRC are that it reduces the bandwidth and I/Os required for repair reads over prior codes,
while still allowing a significant reduction in storage overhead. Intel ISA library also supports
LRC in its update and can also be leveraged. The implementation would also consider how to
distribute the calculating of local and global parity blocks to other relevant DataNodes.

This message was sent by Atlassian JIRA

View raw message