hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6425) reset postponedMisreplicatedBlocks and postponedMisreplicatedBlocksCount when NN becomes active
Date Sat, 17 May 2014 05:44:14 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ming Ma updated HDFS-6425:
--------------------------

    Status: Patch Available  (was: Open)

> reset postponedMisreplicatedBlocks and postponedMisreplicatedBlocksCount when NN becomes
active
> -----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6425
>                 URL: https://issues.apache.org/jira/browse/HDFS-6425
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: HDFS-6425.patch
>
>
> Sometimes we have large number of over replicates when NN fails over. When the new active
NN took over, over replicated blocks will be put to postponedMisreplicatedBlocks until all
DNs for that block aren't stale anymore.
> We have a case where NNs flip flop. Before postponedMisreplicatedBlocks became empty,
NN fail over again and again. So postponedMisreplicatedBlocks just kept increasing until the
cluster is stable. 
> In addition, large postponedMisreplicatedBlocks could make rescanPostponedMisreplicatedBlocks
slow. rescanPostponedMisreplicatedBlocks takes write lock. So it could slow down the block
report processing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message