hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-517) Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
Date Mon, 03 Aug 2009 22:45:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738652#action_12738652
] 

Konstantin Shvachko commented on HDFS-517:
------------------------------------------

- {{BlockInfoUnderConstruction}} should be a subclass of {{BlockInfo}}. It should contain
an array of {{locations}} reflecting the write pipeline order, and replica states.
- When a new block is allocated it is created as a {{BlockInfoUnderConstruction}}. When the
block changes its state to {{COMPLETE}} the {{BlockInfoUnderConstruction}} is replaced by
regular {{BlockInfo}} in the blocksMap, the file inode, and the data-node block list.
- Currently we do not maintain replication of blocks under construction so other maps like
{{UnderReplicatedBlocks}} or {{excessReplicateMap}} would not contain these blocks and therefore
do need to be updated.
- {{BlockInfoUnderConstruction.locations}} should be used to maintain the pipeline and for
block recovery, but not for reading. So locations remain invisible for readers until the replicas
are reported by data-nodes. This is consistent with current behavior, and should change later
when the new read is implemented.
- When a replica is reported by the data-node the block is included into the list of blocks
belonging to the corresponding {{DatanodeDescriptor}} and thus becomes readable.
- The {{locations}} and {{replicaStates}} are not persistent fields (consistent with current
implementation). When name-node restarts it sets the block state to {{UNDER_RECOVERY}} and
waits for block recovery according to the append design.
- {{INodeFileUnderConstruction.targets}} field becomes redundant.


> Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-517
>                 URL: https://issues.apache.org/jira/browse/HDFS-517
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: Append Branch
>
>
> Currently when a block is created its locations are stored in {{INodeFileUnderConstruction.targets}},
which correspond to the last allocated block. With the new append design we will need to keep
track of block replicas for several blocks rather than just the last one.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message