hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11543) Test multiple erasure coding implementations
Date Fri, 17 Mar 2017 18:31:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930438#comment-15930438
] 

Andrew Wang commented on HDFS-11543:
------------------------------------

Is this related to HDFS-11066 and HDFS-9962? I agree that it'd be really nice to parameterize
the tests for all the different EC coder schemas and coder implementations.

> Test multiple erasure coding implementations
> --------------------------------------------
>
>                 Key: HDFS-11543
>                 URL: https://issues.apache.org/jira/browse/HDFS-11543
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha2
>            Reporter: László Bence Nagy
>            Priority: Minor
>              Labels: test
>
> Potentially, multiple native erasure coding plugins will be available to be used from
HDFS later on. These plugins should be tested as well. For example, the *NativeRSRawErasureCoderFactory*
class - which is used for instantiating the native ISA-L plugin's encoder and decoder objects
- are used in 5 test files under the *hadoop/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/*
directory. The files are:
> - *TestDFSStripedInputStream.java*
> - *TestDFSStripedOutputStream.java*
> - *TestDFSStripedOutputStreamWithFailure.java*
> - *TestReconstructStripedFile.java*
> - *TestUnsetAndChangeDirectoryEcPolicy.java*
> Other erasure coding plugins should be tested in these cases as well in a nice way (not
by for example making a new file for every new erasure coding plugin). For this purpose [parameterized
tests|https://github.com/junit-team/junit4/wiki/parameterized-tests] might be used.
> This is also true for the *hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/rawcoder/*
directory where this approach could be used for example for the interoperability tests (when
it is checked that certain erasure coding implementations are compatible with each other by
doing the encoding and decoding operations with different plugins and verifying their results).
The plugin pairs which should be tested could be the parameters for the parameterized tests.
> The parameterized test is just an idea, there can be other solutions as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message