Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Date: Tue, 13 Nov 2012 02:15:14 +0000 (UTC)
From: "Kihwal Lee (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <1893534818.104823.1352772914610.JavaMail.jiratomcat@arcas>
Subject: [jira] [Created] (HDFS-4183) Throttle block recovery
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Kihwal Lee created HDFS-4183:
--------------------------------

             Summary: Throttle block recovery
                 Key: HDFS-4183
                 URL: https://issues.apache.org/jira/browse/HDFS-4183
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: name-node
    Affects Versions: 2.0.2-alpha, 0.23.4
            Reporter: Kihwal Lee
            Priority: Critical


When a large number of files are abandoned without closing, a storm of lease expiration follows in about an hour (lease hard limit). For the last block of each file, block recovery is initiated and when the datanode is done, it calls commitBlockSynchronization() is called against namenode. A burst of these calls can slow down namenode considerably. We need to throttle block recovery and/or speed up the rate at which commitBlockSynchronization() is served.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira