hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leitao Guo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7692) BlockPoolSliceStorage#loadBpStorageDirectories(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories.
Date Wed, 28 Jan 2015 10:56:34 GMT
Leitao Guo created HDFS-7692:
--------------------------------

             Summary: BlockPoolSliceStorage#loadBpStorageDirectories(...) should support MultiThread
to speedup the upgrade of block pool at multi storage directories.
                 Key: HDFS-7692
                 URL: https://issues.apache.org/jira/browse/HDFS-7692
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: datanode
    Affects Versions: 2.5.2
            Reporter: Leitao Guo


{code:title=BlockPoolSliceStorage#loadBpStorageDirectories(...)|borderStyle=solid}
for (File dataDir : dataDirs) {
        if (containsStorageDir(dataDir)) {
          throw new IOException(
              "BlockPoolSliceStorage.recoverTransitionRead: " +
                  "attempt to load an used block storage: " + dataDir);
        }
        StorageDirectory sd =
            loadStorageDirectory(datanode, nsInfo, dataDir, startOpt);
        succeedDirs.add(sd);
      }
{code}

In the above code the storage directories will be analyzed one by one, which is really time
consuming when upgrading HDFS with datanodes have dozens of large volumes.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message