hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rashmi Vinayak (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11828) Implement the Hitchhiker erasure coding algorithm
Date Wed, 10 Feb 2016 05:35:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140359#comment-15140359
] 

Rashmi Vinayak commented on HADOOP-11828:
-----------------------------------------

Hi [~jack_liuquan], [~drankye], [~zhz],

I am super excited to see this being resolved! Thank you all for the efforts that you put
in. I agree with [~zhz] that it would be good to get some performance results comparing RS
and Hitchhiker based on the new implementation. This would guide enterprises who are considering
using erasure coding, and thus leading to a greater impact from this effort and HDFS-EC in
general as they will come to know about this more efficient EC option. 

> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
>                 Key: HADOOP-11828
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11828
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0
>            Reporter: Zhe Zhang
>            Assignee: jack liuquan
>             Fix For: 3.0.0
>
>         Attachments: 7715-hitchhikerXOR-v2-testcode.patch, 7715-hitchhikerXOR-v2.patch,
HADOOP-11828-hitchhikerXOR-V3.patch, HADOOP-11828-hitchhikerXOR-V4.patch, HADOOP-11828-hitchhikerXOR-V5.patch,
HADOOP-11828-hitchhikerXOR-V6.patch, HADOOP-11828-hitchhikerXOR-V7.patch, HADOOP-11828-v8.patch,
HDFS-7715-hhxor-decoder.patch, HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker | http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf]
is a new erasure coding algorithm developed as a research project at UC Berkeley. It has been
shown to reduce network traffic and disk I/O by 25%-45% during data reconstruction while retaining
the same storage capacity and failure tolerance capability as RS codes. This JIRA aims to
introduce Hitchhiker to the HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message