hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs
Date Thu, 28 Jan 2016 07:44:39 GMT
Kai Zheng created HDFS-9719:

             Summary: Refactoring ErasureCodingWorker into smaller reusable constructs
                 Key: HDFS-9719
                 URL: https://issues.apache.org/jira/browse/HDFS-9719
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Kai Zheng
            Assignee: Kai Zheng

This would suggest and refactor {{ErasureCodingWorker}} into smaller constructs to be reused
in other places like block group checksum computing in datanode side. As discussed in HDFS-8430
and implemented in HDFS-9694 patch, checksum computing for striped block groups would be distributed
to datanode in the group, where data block data should be able to be reconstructed when missed/corrupted
to recompute the block checksum. The most needed codes are in the current ErasureCodingWorker
and could be reused in order to avoid duplication. Fortunately, we have very good and complete
tests, which would make the refactoring much easier. The refactoring will also help a lot
for subsequent tasks in phase II for non-striping erasure coded files and blocks. 

This message was sent by Atlassian JIRA

View raw message