hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm
Date Mon, 09 Mar 2015 21:54:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353702#comment-14353702

Zhe Zhang commented on HDFS-7715:

Some questions from understanding the current Hitchhiker implementation:
# Is jointEncodeSubPacketsNew() in PiggyBackCode class the implemention of Hitchhiker-XOR+
and Hitchhiker-nonXOR ?
# Is generatorMatrixForParity used for Hitchhiker-nonXOR? Why not used the default Vandermonde
matrix ? Is there any special characteristics in generatorMatrixForParity ?
# generatorMatrixForParity is hard coding for (10,4). for Generalization to any (k, r), how
can I modify the generatorMatrixForParity?

> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>                 Key: HDFS-7715
>                 URL: https://issues.apache.org/jira/browse/HDFS-7715
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: jack liuquan
> [Hitchhiker | http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf]
is a new erasure coding algorithm developed as a research project at UC Berkeley. It has been
shown to reduce network traffic and disk I/O by 25% and 45% during data reconstruction. This
JIRA aims to introduce Hitchhiker to the HDFS-EC framework, as one of the pluggable codec
> The existing implementation is based on HDFS-RAID. 

This message was sent by Atlassian JIRA

View raw message