hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9275) Fix TestRecoverStripedFile
Date Tue, 27 Oct 2015 11:13:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976233#comment-14976233

Yi Liu commented on HDFS-9275:

Walter, sorry I forgot this JIRA :-) 

For continuous block, if n replicas are missed (for total 3 replicas, at most 2 can be missed,
so n <3), we will check the total of replicas in PendingReplicationBlocks to see whether
we need to schedule new block replication.
For block reconstruction of striped block, ideally we should follow this,  for any missed
striped internal block, we just need to reconstruct 1, so we should check whether there is
1 in pendingReplicationBlocks,  but currently we track the block group in the list.  Then
it becomes we compare the total missed striped internal blocks with the number in PendingReplicationBlocks,
if there are more than two missed striped internal blocks and one is reconstructed first,
then there may be some unnecessary reconstruction.   I think we can do a simple improvement
for striped block, if there is one in PendingReplicationBlocks, then we don't schedule new
reconstruction work instead of comparing the number of missed striped internal blocks. 

> Fix TestRecoverStripedFile
> --------------------------
>                 Key: HDFS-9275
>                 URL: https://issues.apache.org/jira/browse/HDFS-9275
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-9275.01.patch, HDFS-9275.02.patch, HDFS-9275.03.patch

This message was sent by Atlassian JIRA

View raw message