hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HDFS-13284) Adjust criteria for LowRedundancyBlocks.QUEUE_VERY_LOW_REDUNDANCY
Date Thu, 15 Mar 2018 18:51:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400955#comment-16400955
] 

Íñigo Goiri commented on HDFS-13284:
------------------------------------

Thanks [~lukmajercak], this makes sense; I would push this.
[~stevel@apache.org], you were the committer for HDFS-2485 which touched this code but after
looking more closely that was a refactor.


> Adjust criteria for LowRedundancyBlocks.QUEUE_VERY_LOW_REDUNDANCY
> -----------------------------------------------------------------
>
>                 Key: HDFS-13284
>                 URL: https://issues.apache.org/jira/browse/HDFS-13284
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs, namenode
>            Reporter: Lukas Majercak
>            Assignee: Lukas Majercak
>            Priority: Major
>         Attachments: HDFS-13284.000.patch
>
>
> LowRedundancyBlocks currently has 5 priority queues:
> QUEUE_HIGHEST_PRIORITY = 0                         - reserved for last replica
blocks
>  QUEUE_VERY_LOW_REDUNDANCY = 1             - *if ((curReplicas * 3) < expectedReplicas)*
>  QUEUE_LOW_REDUNDANCY = 2                         - the rest
>  QUEUE_REPLICAS_BADLY_DISTRIBUTED = 3
>  QUEUE_WITH_CORRUPT_BLOCKS = 4
> The problem lies in  QUEUE_VERY_LOW_REDUNDANCY. Currently, a block that has curReplicas=2
and expectedReplicas=4 is treated the same as a block with curReplicas=3 and expectedReplicas=4.
A block with 2/3 replicas is also put into QUEUE_LOW_REDUNDANCY. 
> The proposal is to change the *{{if ((curReplicas * 3) < expectedReplicas)}}* check
to *{{if ((curReplicas * 2) <= expectedReplicas || curReplicas == 2)}}*
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message