hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-2184) Revisit Namenode locking
Date Fri, 22 Jul 2011 06:52:58 GMT
Revisit Namenode locking
------------------------

                 Key: HDFS-2184
                 URL: https://issues.apache.org/jira/browse/HDFS-2184
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: name-node
            Reporter: Eli Collins


While working on HDFS-988 I noticed that the locking in FSNamesystem and FSDirectory could
be improved. Some observations:

The namesystem lock (fsLock) is always taken before acquiring the directory lock (dirLock).
Therefore the only time when the directory lock is needed is when the fsLock is taken for
reading and the directory lock is taken for writing, but I don't think that ever happens.
Therefore we can probably get rid of the directory lock.

In HDFS-988 I modified handleHeartbeat to take the read lock so it's synchronized with register
datanode. I also added a missing synchronization of datanodeMap to wipeDatanode because handleHeartbeat
calls getDatanode() while only holding locks on heartbeats and datanodeMap, but registerDatanode
mutates datanodeMap without locking either. We should revisit which locks/synchronization
protect which data structures, there may be other similar bugs and also opportunities to increase
parallelism.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message