hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-3767) Finer grained locking in DN
Date Mon, 06 Aug 2012 19:58:02 GMT
Todd Lipcon created HDFS-3767:
---------------------------------

             Summary: Finer grained locking in DN
                 Key: HDFS-3767
                 URL: https://issues.apache.org/jira/browse/HDFS-3767
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: performance
    Affects Versions: 3.0.0
            Reporter: Todd Lipcon


In testing a high-write-throughput workload, I see the DN maintain good performance most of
the time, except that occasionally one thread will block for a few seconds in {{finalizeReplica}}.
It does so holding the FSDatasetImpl lock, which causes all other writer threads to block
behind it. HDFS-1148 (making it a rw lock) would help here, but a bigger help would be to
go do finer-grained locking (eg per block or per-subdir).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message