hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lukas Majercak (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13284) Adjust criteria for LowRedundancyBlocks.QUEUE_VERY_LOW_REDUNDANCY
Date Fri, 16 Mar 2018 17:47:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402253#comment-16402253
] 

Lukas Majercak commented on HDFS-13284:
---------------------------------------

Added patch 001 for fixing checkstyle. Unit test failures seem unrelated.

> Adjust criteria for LowRedundancyBlocks.QUEUE_VERY_LOW_REDUNDANCY
> -----------------------------------------------------------------
>
>                 Key: HDFS-13284
>                 URL: https://issues.apache.org/jira/browse/HDFS-13284
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs, namenode
>            Reporter: Lukas Majercak
>            Assignee: Lukas Majercak
>            Priority: Major
>         Attachments: HDFS-13284.000.patch, HDFS-13284.001.patch
>
>
> LowRedundancyBlocks currently has 5 priority queues:
> QUEUE_HIGHEST_PRIORITY = 0                         - reserved for last replica
blocks
>  QUEUE_VERY_LOW_REDUNDANCY = 1             - *if ((curReplicas * 3) < expectedReplicas)*
>  QUEUE_LOW_REDUNDANCY = 2                         - the rest
>  QUEUE_REPLICAS_BADLY_DISTRIBUTED = 3
>  QUEUE_WITH_CORRUPT_BLOCKS = 4
> The problem lies in  QUEUE_VERY_LOW_REDUNDANCY. Currently, a block that has curReplicas=2
and expectedReplicas=4 is treated the same as a block with curReplicas=3 and expectedReplicas=4.
A block with 2/3 replicas is also put into QUEUE_LOW_REDUNDANCY. 
> The proposal is to change the *{{if ((curReplicas * 3) < expectedReplicas)}}* check
to *{{if ((curReplicas * 2) <= expectedReplicas || curReplicas == 2)}}*
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message