hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jack liuquan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm
Date Sat, 28 Mar 2015 08:17:53 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

jack liuquan updated HDFS-7715:
-------------------------------
    Attachment: 7715-hitchhikerXOR-v2.patch

Hi kai,
I have uploaded a new implementation of hitchhiker-XOR version. This separate the RS and HH
specific math calculation, reusing the existing RS code as more as possible. Hope native XOR
and RS raw coders can be utilized to benefit from the performance improvement.
please review the code.Thanks!

> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
>                 Key: HDFS-7715
>                 URL: https://issues.apache.org/jira/browse/HDFS-7715
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: jack liuquan
>         Attachments: 7715-hitchhikerXOR-v2.patch, HDFS-7715-hhxor-decoder.patch, HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker | http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf]
is a new erasure coding algorithm developed as a research project at UC Berkeley. It has been
shown to reduce network traffic and disk I/O by 25%-45% during data reconstruction. This JIRA
aims to introduce Hitchhiker to the HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message