hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11542) Raw Reed-Solomon coder in pure Java
Date Mon, 09 Feb 2015 19:51:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312739#comment-14312739

Zhe Zhang commented on HADOOP-11542:

Thanks Kai for the updated patch. 

# Did you highlight the changes from HDFS-RAID's {{GaloisField}} class? Based on your description,
should I look for all {{ByteBuffer}} usage in your new class, like the below?
+  /**
+   * A "bulk" version of the solveVandermondeSystem, using ByteBuffer.
+   */
+  public void solveVandermondeSystem(int[] x, ByteBuffer[] y,
+                                     int len, int dataLen) {
# Otherwise, the code structure looks good to me. However it's hard to fully understand and
verify the arithmetic logics so I think we should be careful to check test results.

# {{testCoding#usingDirectBuffer}} is not used?
# {{testCoding}} actually only tested lost data chunks; do we plan to test lost parity chunks
as well?
# In {{prepareInputChunksForDecoding}}, why are parity chunks put in front of data chunks?
Functionally it doesn't really matter, but it'd be better to have a consistent ordering.
# {{TestJRSRawCoder#prepare}} looks it belongs to one of the base classes -- either {{TestCoderBase}}
or {{TestRawCoderBase}}?
# "Please note this implementation doesn't outperform" -- let's remove this and just say "Please
always use the native implementation when possible."
# {{eraseDataFromChunk}} could used bulk put instead of putting individual bytes.
# {{TestCoderBase#compareAndVerify}} could use {{deepEquals}} for nested arrays
# It doesn't actually matter in a loop condition, but would be better to unify to {{i++}}.
for (int i = 0; i < erasedChunks.length; ++i) {

> Raw Reed-Solomon coder in pure Java
> -----------------------------------
>                 Key: HADOOP-11542
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11542
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: HDFS-EC
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HADOOP-11542-v1.patch, HADOOP-11542-v2.patch, HADOOP-11542-v3.patch
> This will implement RS coder by porting existing codes in HDFS-RAID in the new codec
and coder framework, which could be useful in case native support isn't available or convenient
in some environments or platforms.

This message was sent by Atlassian JIRA

View raw message