hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-517) Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
Date Mon, 03 Aug 2009 22:45:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738652#action_12738652

Konstantin Shvachko commented on HDFS-517:

- {{BlockInfoUnderConstruction}} should be a subclass of {{BlockInfo}}. It should contain
an array of {{locations}} reflecting the write pipeline order, and replica states.
- When a new block is allocated it is created as a {{BlockInfoUnderConstruction}}. When the
block changes its state to {{COMPLETE}} the {{BlockInfoUnderConstruction}} is replaced by
regular {{BlockInfo}} in the blocksMap, the file inode, and the data-node block list.
- Currently we do not maintain replication of blocks under construction so other maps like
{{UnderReplicatedBlocks}} or {{excessReplicateMap}} would not contain these blocks and therefore
do need to be updated.
- {{BlockInfoUnderConstruction.locations}} should be used to maintain the pipeline and for
block recovery, but not for reading. So locations remain invisible for readers until the replicas
are reported by data-nodes. This is consistent with current behavior, and should change later
when the new read is implemented.
- When a replica is reported by the data-node the block is included into the list of blocks
belonging to the corresponding {{DatanodeDescriptor}} and thus becomes readable.
- The {{locations}} and {{replicaStates}} are not persistent fields (consistent with current
implementation). When name-node restarts it sets the block state to {{UNDER_RECOVERY}} and
waits for block recovery according to the append design.
- {{INodeFileUnderConstruction.targets}} field becomes redundant.

> Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
> -----------------------------------------------------------------------------------
>                 Key: HDFS-517
>                 URL: https://issues.apache.org/jira/browse/HDFS-517
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: Append Branch
> Currently when a block is created its locations are stored in {{INodeFileUnderConstruction.targets}},
which correspond to the last allocated block. With the new append design we will need to keep
track of block replicas for several blocks rather than just the last one.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message