hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-4183) Throttle block recovery
Date Thu, 30 Jan 2014 21:38:11 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kihwal Lee resolved HDFS-4183.
------------------------------

          Resolution: Duplicate
    Target Version/s: 2.1.0-beta, 3.0.0  (was: 3.0.0, 2.1.0-beta)

> Throttle block recovery
> -----------------------
>
>                 Key: HDFS-4183
>                 URL: https://issues.apache.org/jira/browse/HDFS-4183
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 0.23.4, 2.0.2-alpha
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Critical
>
> When a large number of files are abandoned without closing, a storm of lease expiration
follows in about an hour (lease hard limit). For the last block of each file, block recovery
is initiated and when the datanode is done, it calls commitBlockSynchronization() is called
against namenode. A burst of these calls can slow down namenode considerably. We need to throttle
block recovery and/or speed up the rate at which commitBlockSynchronization() is served.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message