hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7912) Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks
Date Tue, 17 Mar 2015 00:57:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364334#comment-14364334
] 

Jing Zhao commented on HDFS-7912:
---------------------------------

Thanks for the review, Zhe!

bq. My only concern is that it adds some getStoredBlock calls

The patch adds {{getStoredBlock}} call in mainly two places: {{removeStoredBlock}} and {{addBlock}}.
The {{addBlock}} function is currently called to handle a "RECEIVED_BLOCK" msg in incremental
block report, and it is not happening when handling a full block report. Thus I guess here
we will not bring too much extra overhead to the NN.

bq. do we need storedBlock in the following code

Here my thought is that we should use BlockInfo for {{countNodes}}, since the function tries
to retrieve the storages from the blocksMap for the given block. And with this we need to
make the change in NameNodeFsck.

> Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7912
>                 URL: https://issues.apache.org/jira/browse/HDFS-7912
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-7912.000.patch
>
>
> Now with striped blocks and the design that uses a single BlockInfoStriped object to
track all the corresponding blocks, we need to clearly distinguish the type Block and BlockInfo
in BlockManager. Specifically, data structures like {{UnderReplicatedBlocks}} and {{PendingReplicationBlocks}}
should track BlockInfo instead of Block in order to support striped block recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message