hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9261) Erasure Coding: Skip encoding the data cells if all the parity data streamers are failed for the current block group
Date Tue, 27 Oct 2015 07:46:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14975896#comment-14975896
] 

Rakesh R commented on HDFS-9261:
--------------------------------

Thanks [~umamaheswararao] for the advice. {{TestDFSStripedOutputStreamWithFailure#testMultipleDatanodeFailure56}}
-> dnIndexSuite 6, 7, 8, is covering all parity datanodes failures.

I've done minor updates. Attached new patch including debugging message. Please review it
again!

> Erasure Coding: Skip encoding the data cells if all the parity data streamers are failed
for the current block group
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9261
>                 URL: https://issues.apache.org/jira/browse/HDFS-9261
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>            Priority: Minor
>         Attachments: HDFS-9261-00.patch, HDFS-9261-01.patch
>
>
> {{DFSStripedOutputStream}} will continue writing with minimum number (dataBlockNum) of
live datanodes. It won't replace the failed datanodes immediately for the current block group.
Consider a case where all the parity data streamers are failed, now it is unnecessary to encode
the data block cells and generate the parity data. This is a corner case where it can skip
{{writeParityCells()}} step.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message