hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9833) Erasure coding: recomputing block checksum on the fly by reconstructing missed/corrupt block
Date Fri, 19 Feb 2016 06:35:19 GMT
Kai Zheng created HDFS-9833:
-------------------------------

             Summary: Erasure coding: recomputing block checksum on the fly by reconstructing
missed/corrupt block
                 Key: HDFS-9833
                 URL: https://issues.apache.org/jira/browse/HDFS-9833
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Kai Zheng
            Assignee: Kai Zheng


As discussed in HDFS-8430 and HDFS-9694, to compute striped file checksum even some of striped
blocks are missed, we need to consider recomputing block checksum on the fly for the missed/corrupt
blocks. To recompute the block checksum, the block data needs to be reconstructed by erasure
decoding, and the main needed codes for the block reconstruction could be borrowed from HDFS-9719,
the refactoring of the existing {{ErasureCodingWorker}}. In EC worker, reconstructed blocks
need to be written out to target datanodes, but here in this case, the remote writing isn't
necessary, as the reconstructed block data is only used to recompute the checksum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message