hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7354) Support parity blocks in block management
Date Tue, 04 Nov 2014 19:02:33 GMT
Zhe Zhang created HDFS-7354:

             Summary: Support parity blocks in block management
                 Key: HDFS-7354
                 URL: https://issues.apache.org/jira/browse/HDFS-7354
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Zhe Zhang
            Assignee: Zhe Zhang

In the current block management system, each block is associated with a file. "Orphan" blocks
are considered corrupt and will be removed.

In this JIRA we extend {{Block}} with a binary flag denoting whether it is a parity block
({{isParity}}). Parity blocks are created, stored, and reported the same way as raw ones.
They have regular block IDs which are unrelated to those of the raw blocks in the same group;
their replicas (normally only 1) are stored in RBW and finalized directories on the DataNode
depending on the stage; they are also included in block reports. The only distinction of a
parity block is the lack of file affiliation. The block management system will be aware of
parity blocks and will _not_ try to remove them.

This message was sent by Atlassian JIRA

View raw message