hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11542) Raw Reed-Solomon coder in pure Java
Date Mon, 09 Feb 2015 19:51:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312739#comment-14312739
] 

Zhe Zhang commented on HADOOP-11542:
------------------------------------

Thanks Kai for the updated patch. 

Structures:
# Did you highlight the changes from HDFS-RAID's {{GaloisField}} class? Based on your description,
should I look for all {{ByteBuffer}} usage in your new class, like the below?
{code}
+  /**
+   * A "bulk" version of the solveVandermondeSystem, using ByteBuffer.
+   */
+  public void solveVandermondeSystem(int[] x, ByteBuffer[] y,
+                                     int len, int dataLen) {
{code}
# Otherwise, the code structure looks good to me. However it's hard to fully understand and
verify the arithmetic logics so I think we should be careful to check test results.

Nits:
# {{testCoding#usingDirectBuffer}} is not used?
# {{testCoding}} actually only tested lost data chunks; do we plan to test lost parity chunks
as well?
# In {{prepareInputChunksForDecoding}}, why are parity chunks put in front of data chunks?
Functionally it doesn't really matter, but it'd be better to have a consistent ordering.
# {{TestJRSRawCoder#prepare}} looks it belongs to one of the base classes -- either {{TestCoderBase}}
or {{TestRawCoderBase}}?
# "Please note this implementation doesn't outperform" -- let's remove this and just say "Please
always use the native implementation when possible."
# {{eraseDataFromChunk}} could used bulk put instead of putting individual bytes.
# {{TestCoderBase#compareAndVerify}} could use {{deepEquals}} for nested arrays
# It doesn't actually matter in a loop condition, but would be better to unify to {{i++}}.
{code}
for (int i = 0; i < erasedChunks.length; ++i) {
{code}

> Raw Reed-Solomon coder in pure Java
> -----------------------------------
>
>                 Key: HADOOP-11542
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11542
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: HDFS-EC
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HADOOP-11542-v1.patch, HADOOP-11542-v2.patch, HADOOP-11542-v3.patch
>
>
> This will implement RS coder by porting existing codes in HDFS-RAID in the new codec
and coder framework, which could be useful in case native support isn't available or convenient
in some environments or platforms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message