hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From László Bence Nagy (JIRA) <j...@apache.org>
Subject [jira] [Created] (HDFS-11543) Test multiple erasure coding implementations
Date Fri, 17 Mar 2017 16:10:42 GMT
László Bence Nagy created HDFS-11543:

             Summary: Test multiple erasure coding implementations
                 Key: HDFS-11543
                 URL: https://issues.apache.org/jira/browse/HDFS-11543
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha2
            Reporter: László Bence Nagy
            Priority: Minor

Potentially, multiple native erasure coding plugins will be available to be used from HDFS
later on. These plugins should be tested as well. For example, the *NativeRSRawErasureCoderFactory*
class - which is used for instantiating the native ISA-L plugin's encoder and decoder objects
- are used in 5 test files under the *hadoop/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/*
directory. The files are:
- *TestDFSStripedInputStream.java*
- *TestDFSStripedOutputStream.java*
- *TestDFSStripedOutputStreamWithFailure.java*
- *TestReconstructStripedFile.java*
- *TestUnsetAndChangeDirectoryEcPolicy.java*

Other erasure coding plugins should be tested in these cases as well in a nice way (not by
for example making a new file for every new erasure coding plugin). For this purpose [parameterized
tests|https://github.com/junit-team/junit4/wiki/parameterized-tests] might be used.

This is also true for the *hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/rawcoder/*
directory where this approach could be used for example for the interoperability tests (when
it is checked that certain erasure coding implementations are compatible with each other by
doing the encoding and decoding operations with different plugins and verifying their results).
The plugin pairs which should be tested could be the parameters for the parameterized tests.

The parameterized test is just an idea, there can be other solutions as well.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message