From hdfs-issues-return-108446-apmail-hadoop-hdfs-issues-archive=hadoop.apache.org@hadoop.apache.org Sat Feb 7 07:13:35 2015 Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1F43B17A9C for ; Sat, 7 Feb 2015 07:13:35 +0000 (UTC) Received: (qmail 25478 invoked by uid 500); 7 Feb 2015 07:13:35 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 25422 invoked by uid 500); 7 Feb 2015 07:13:35 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 25411 invoked by uid 99); 7 Feb 2015 07:13:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 Feb 2015 07:13:34 +0000 Date: Sat, 7 Feb 2015 07:13:34 +0000 (UTC) From: "Leitao Guo (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-7692) DataStorage#addStorageLocations(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310592#comment-14310592 ] Leitao Guo commented on HDFS-7692: ---------------------------------- [~szetszwo] thanks for your comments, I will update the patch asap. > DataStorage#addStorageLocations(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories. > ---------------------------------------------------------------------------------------------------------------------------------- > > Key: HDFS-7692 > URL: https://issues.apache.org/jira/browse/HDFS-7692 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode > Affects Versions: 2.5.2 > Reporter: Leitao Guo > Assignee: Leitao Guo > Attachments: HDFS-7692.01.patch > > > {code:title=DataStorage#addStorageLocations(...)|borderStyle=solid} > for (StorageLocation dataDir : dataDirs) { > File root = dataDir.getFile(); > ... ... > bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs, startOpt); > addBlockPoolStorage(bpid, bpStorage); > ... ... > successVolumes.add(dataDir); > } > {code} > In the above code the storage directories will be analyzed one by one, which is really time consuming when upgrading HDFS with datanodes have dozens of large volumes. MultiThread dataDirs analyzing should be supported here to speedup upgrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)