hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12013) Generate fixed data to perform erasure coder test
Date Thu, 21 May 2015 19:22:17 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554906#comment-14554906

Zhe Zhang commented on HADOOP-12013:

Thanks Kai for the work. The overall structure of allowing both fixed and random encoding
inputs looks good.

My main question is how {{FIXED_DATA_GENERATOR}} leads to the desired repeatable behavior.
It is a static variable of the base testing class, so the generated byte is not really very
predictable. In {{SimulatedFSDataset}}, the simulated byte is predictable because it can be
statically calculated from the block ID and offset.

Do you have an example of how to use it in a real test? Like setting an expected value and
assert it (I assume the assertion should fail without the fixed value but will pass with it).

Nit: the below code can be replaced by a byte mask, similar to {{SimulatedFSDataset#simulatedByte}}:

+      buffer[i] = (byte) FIXED_DATA_GENERATOR++;
+      if (FIXED_DATA_GENERATOR == 256) {
+      }
+    }

> Generate fixed data to perform erasure coder test
> -------------------------------------------------
>                 Key: HADOOP-12013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12013
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>             Fix For: HDFS-7285
>         Attachments: HADOOP-12013-HDFS-7285-v1.patch
> While working on native erasure coders, it was found useful to allow generating and using
fixed data to test raw erasure coders to ease the debugging some coding issues.

This message was sent by Atlassian JIRA

View raw message