hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11503) Integrate Chocolate Cloud RS coder implementation
Date Tue, 04 Apr 2017 21:03:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955798#comment-15955798

Andrew Wang commented on HDFS-11503:

Hi Marcell,

The licensing decision is up to your company. Personally I think having a free version is
nice so users can try it out, but that doesn't affect whether we commit the integration code
to Apache Hadoop. We can't distribute a proprietary library as part of a Hadoop release, so
it doesn't make sense for us to maintain the integration code for that library in our tree.

This situation is exactly why we are working on pluggable EC policies and coders. This lets
you provide the integration code and coder library at runtime.

We'd be happy to work with you to refine the pluggable interfaces to meet your needs. Having
an additional implementation (particularly an out-of-tree one) would be great for validation
and testing.

> Integrate Chocolate Cloud RS coder implementation
> -------------------------------------------------
>                 Key: HDFS-11503
>                 URL: https://issues.apache.org/jira/browse/HDFS-11503
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Andrew Wang
>            Assignee: Marcell Feher
>         Attachments: HDFS-11503.patch
> Quote from Marcell on HDFS-7285:
> First of all let me introduce ourselves: we are Chocolate Cloud from Denmark, we use
erasure coding to improve storage solutions. We already have Reed-Solomon and Random Linear
Network Coding backends for Liberasurecode, and now we are at the final stage of developing
our RS plugin to HDFS-EC. The performance of our plugin is similar to ISA-L's, in some configurations
we are better, in others we are worse (our initial speed comparison charts can be found here:
> We would like our plugin to become officially supported in Hadoop 3.0. We can already
provide a preliminary version of our (native) library and a patch with the necessary glue
code for the next alpha release.
> I'd like to know your thoughts about whether it's possible and how it could be achieved.
> P.S: I'm happy to share more details if there's interest

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message