Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8F8E8176BA for ; Fri, 30 Jan 2015 21:43:45 +0000 (UTC) Received: (qmail 50066 invoked by uid 500); 30 Jan 2015 21:43:45 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 49799 invoked by uid 500); 30 Jan 2015 21:43:45 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 49773 invoked by uid 99); 30 Jan 2015 21:43:45 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Jan 2015 21:43:45 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 9747FE0100; Fri, 30 Jan 2015 21:43:45 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zhz@apache.org To: common-commits@hadoop.apache.org Date: Fri, 30 Jan 2015 21:43:47 -0000 Message-Id: In-Reply-To: <057d332d65304d0ca22cc3456f016a22@git.apache.org> References: <057d332d65304d0ca22cc3456f016a22@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/9] hadoop git commit: HDFS-7603. The background replication queue initialization may not let others run. Contributed by Kihwal Lee. HDFS-7603. The background replication queue initialization may not let others run. Contributed by Kihwal Lee. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/0c151901 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/0c151901 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/0c151901 Branch: refs/heads/HDFS-EC Commit: 0c15190148a7f0d970432708f3c8a13a0412d33b Parents: 570914e Author: Kihwal Lee Authored: Thu Jan 29 12:36:22 2015 -0600 Committer: Zhe Zhang Committed: Fri Jan 30 13:42:04 2015 -0800 ---------------------------------------------------------------------- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++ .../apache/hadoop/hdfs/server/blockmanagement/BlockManager.java | 5 +++++ 2 files changed, 8 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/0c151901/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt index 4932c80..73f175f 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt @@ -561,6 +561,9 @@ Release 2.7.0 - UNRELEASED HDFS-7675. Remove unused member DFSClient#spanReceiverHost (cmccabe) + HDFS-7603. The background replication queue initialization may not let + others run (kihwal) + OPTIMIZATIONS HDFS-7454. Reduce memory footprint for AclEntries in NameNode. http://git-wip-us.apache.org/repos/asf/hadoop/blob/0c151901/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java index f240569..52bca48 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java @@ -2612,6 +2612,9 @@ public class BlockManager { long totalBlocks = blocksMap.size(); replicationQueuesInitProgress = 0; long totalProcessed = 0; + long sleepDuration = + Math.max(1, Math.min(numBlocksPerIteration/1000, 10000)); + while (namesystem.isRunning() && !Thread.currentThread().isInterrupted()) { int processed = 0; namesystem.writeLockInterruptibly(); @@ -2668,6 +2671,8 @@ public class BlockManager { } } finally { namesystem.writeUnlock(); + // Make sure it is out of the write lock for sufficiently long time. + Thread.sleep(sleepDuration); } } if (Thread.currentThread().isInterrupted()) {