hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11542) Raw Reed-Solomon coder in pure Java
Date Mon, 09 Feb 2015 22:41:36 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313053#comment-14313053
] 

Kai Zheng commented on HADOOP-11542:
------------------------------------

bq.testCoding actually only tested lost data chunks; do we plan to test lost parity chunks
as well?
Yes indeed. It's planned to have follow up a JIRA to enhance the tests for parity chunks as
well. Like {{erasedDataIndexes}}, {{erasedParityIndexes}} will be added to specify which parity
units are to be erased and recovered then.
bq.In prepareInputChunksForDecoding, why are parity chunks put in front of data chunks? Functionally
it doesn't really matter, but it'd be better to have a consistent ordering.
I thought the consistent ordering should be: parity units + data units, which respects the
implementation of {{GaloisField}}. I will make sure other coders like {{IsaRSRawCoder}} will
also follow that.

> Raw Reed-Solomon coder in pure Java
> -----------------------------------
>
>                 Key: HADOOP-11542
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11542
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: HDFS-EC
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HADOOP-11542-v1.patch, HADOOP-11542-v2.patch, HADOOP-11542-v3.patch,
HDFS-RAID-GaloisField.java
>
>
> This will implement RS coder by porting existing codes in HDFS-RAID in the new codec
and coder framework, which could be useful in case native support isn't available or convenient
in some environments or platforms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message