hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-1093) Improve namenode scalability by splitting the FSNamesystem synchronized section in a read/write lock
Date Sun, 12 Sep 2010 09:25:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur updated HDFS-1093:
-----------------------------------

    Attachment: NNreadwriteLock_trunk_4.txt

I implemented suresh's suggestion regarding FSPermissionChecker.checkPermission.

Regarding the readLock on getBlockLocationsInternal: access times are precise upto an hour
boundary. In many cases, the getBlockLocationsInternal does not have to update the accesstime
(even though access time is enabled). That is the reason for the dance to try first with readLock,
and if an access time update is required, only then acquire the writeLock.


> Improve namenode scalability by splitting the FSNamesystem synchronized section in a
read/write lock
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1093
>                 URL: https://issues.apache.org/jira/browse/HDFS-1093
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: NNreadwriteLock.txt, NNreadwriteLock_trunk_1.txt, NNreadwriteLock_trunk_2.txt,
NNreadwriteLock_trunk_3.txt, NNreadwriteLock_trunk_4.txt
>
>
> Most critical data structures in the NameNode (NN) are protected by a syncronized methods
in the FSNamesystem class. This essentially makes critical code paths in the NN single-threaded.
However, a large percentage of the NN calls are listStatus, getBlockLocations, etc which do
not change internal data structures at all, these are read-only calls. If we change the FSNamesystem
lock to a read/write lock, many of the above operations can occur in parallel, thus improving
the scalability of the NN.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message