hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5453) Support fine grain locking in FSNamesystem
Date Mon, 23 Dec 2013 20:36:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855918#comment-13855918
] 

Suresh Srinivas commented on HDFS-5453:
---------------------------------------

[~ebortnik], thanks for the spreadsheet.

In the spreadsheet, I assume BASE is with the existing locking and ASYNC is with all synchronization
removed (is that what you mean by lock-free?). There are two synchronizations that happen,
FSNamesystem and editlog related. Can you describe if you removed both of them in lock-free?
Did you have namenode writing to editlog in these benchmarks? Also having an idea on the number
of RPC handlers on the namenode would help.

I am surprised that for "GET BLOCK LOCATIONS", which is a read operation, the data does not
show the benefit of read-write lock. I plan on running some of this benchmark and understand
these numbers. Do you have a patch related to lock-free implementation?


> Support fine grain locking in FSNamesystem
> ------------------------------------------
>
>                 Key: HDFS-5453
>                 URL: https://issues.apache.org/jira/browse/HDFS-5453
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: async_simulation.xlsx
>
>
> The namesystem currently uses a course grain lock to control access.  This prevents concurrent
writers in different branches of the tree, and prevents readers from accessing branches that
writers aren't using.
> Features that introduce latency to namesystem operations, such as cold storage of inodes,
will need fine grain locking to avoid degrading the entire namesystem's throughput.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message