hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-2379) 0.20: Allow block reports to proceed without holding FSDataset lock
Date Wed, 28 Sep 2011 09:04:45 GMT
0.20: Allow block reports to proceed without holding FSDataset lock
-------------------------------------------------------------------

                 Key: HDFS-2379
                 URL: https://issues.apache.org/jira/browse/HDFS-2379
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: data-node
    Affects Versions: 0.20.206.0
            Reporter: Todd Lipcon
            Priority: Critical


As disks are getting larger and more plentiful, we're seeing DNs with multiple millions of
blocks on a single machine. When page cache space is tight, block reports can take multiple
minutes to generate. Currently, during the scanning of the data directories to generate a
report, the FSVolumeSet lock is held. This causes writes and reads to block, timeout, etc,
causing big problems especially for clients like HBase.

This JIRA is to explore some of the ideas originally discussed in HADOOP-4584 for the 0.20.20x
series.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message