hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HDFS-10854) Remove createStripedFile and addBlockToFile by creating real EC files
Date Mon, 31 Oct 2016 05:47:59 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

SammiChen reassigned HDFS-10854:

    Assignee: SammiChen

> Remove createStripedFile and addBlockToFile by creating real EC files
> ---------------------------------------------------------------------
>                 Key: HDFS-10854
>                 URL: https://issues.apache.org/jira/browse/HDFS-10854
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding, test
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Zhe Zhang
>            Assignee: SammiChen
> {{DFSTestUtil#createStripedFile}} and {{addBlockToFile}} were developed before we completed
EC client. They were used to test the {{NameNode}} EC logic when the client was unable to
really create/read/write EC files.
> They are causing confusions in other issues about {{NameNode}}. For example, in one of
the patches under {{HDFS-10301}}, {{testProcessOverReplicatedAndMissingStripedBlock}} fails
because in the test we fake a block report from a DN, with a randomly generated storage ID.
The DN itself is never aware of that storage. This is not possible in a real production environment.
> {code}
>       DatanodeStorage storage = new DatanodeStorage(UUID.randomUUID().toString());
>       StorageReceivedDeletedBlocks[] reports = DFSTestUtil
>           .makeReportForReceivedBlock(block,
>               ReceivedDeletedBlockInfo.BlockStatus.RECEIVING_BLOCK, storage);
>       for (StorageReceivedDeletedBlocks report : reports) {
>         ns.processIncrementalBlockReport(dn.getDatanodeId(), report);
>       }
> {code}
> Now that we have a fully functional EC client, we should remove the old testing logic
and use similar logic as non-EC tests (creating real files and emulate blocks missing / being

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message