Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Fri, 19 Feb 2016 01:44:18 +0000 (UTC)
From: "jack liuquan (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12771443.1422656261000.89062.1455846258162@Atlassian.JIRA>
In-Reply-To: <JIRA.12771443.1422656261000@Atlassian.JIRA>
References: <JIRA.12771443.1422656261000@Atlassian.JIRA>
 <JIRA.12771443.1422656261663@arcas>
Subject: [jira] [Commented] (HADOOP-11828) Implement the Hitchhiker erasure
 coding algorithm
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153512#comment-15153512 ] 

jack liuquan commented on HADOOP-11828:
---------------------------------------

Hi [~rashmikv],
Kai is right, current hitchhiker coders are not couple with specific RS implementation. It picks up the concrete implementation from configuration.
you can see {{HHXORErasureDecoder.java}} for more details.

  private RawErasureDecoder checkCreateRSRawDecoder() {
    if (rsRawDecoder == null) {
      rsRawDecoder = CodecUtil.createRSRawDecoder(getConf(),
              getNumDataUnits(), getNumParityUnits());
    }
    return rsRawDecoder;
  }

Thanks,
Jack

> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
>                 Key: HADOOP-11828
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11828
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0
>            Reporter: Zhe Zhang
>            Assignee: jack liuquan
>             Fix For: 3.0.0
>
>         Attachments: 7715-hitchhikerXOR-v2-testcode.patch, 7715-hitchhikerXOR-v2.patch, HADOOP-11828-hitchhikerXOR-V3.patch, HADOOP-11828-hitchhikerXOR-V4.patch, HADOOP-11828-hitchhikerXOR-V5.patch, HADOOP-11828-hitchhikerXOR-V6.patch, HADOOP-11828-hitchhikerXOR-V7.patch, HADOOP-11828-v8.patch, HDFS-7715-hhxor-decoder.patch, HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker | http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is a new erasure coding algorithm developed as a research project at UC Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45% during data reconstruction while retaining the same storage capacity and failure tolerance capability as RS codes. This JIRA aims to introduce Hitchhiker to the HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)