hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5453) Support fine grain locking in FSNamesystem
Date Mon, 23 Dec 2013 20:36:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855918#comment-13855918

Suresh Srinivas commented on HDFS-5453:

[~ebortnik], thanks for the spreadsheet.

In the spreadsheet, I assume BASE is with the existing locking and ASYNC is with all synchronization
removed (is that what you mean by lock-free?). There are two synchronizations that happen,
FSNamesystem and editlog related. Can you describe if you removed both of them in lock-free?
Did you have namenode writing to editlog in these benchmarks? Also having an idea on the number
of RPC handlers on the namenode would help.

I am surprised that for "GET BLOCK LOCATIONS", which is a read operation, the data does not
show the benefit of read-write lock. I plan on running some of this benchmark and understand
these numbers. Do you have a patch related to lock-free implementation?

> Support fine grain locking in FSNamesystem
> ------------------------------------------
>                 Key: HDFS-5453
>                 URL: https://issues.apache.org/jira/browse/HDFS-5453
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: async_simulation.xlsx
> The namesystem currently uses a course grain lock to control access.  This prevents concurrent
writers in different branches of the tree, and prevents readers from accessing branches that
writers aren't using.
> Features that introduce latency to namesystem operations, such as cold storage of inodes,
will need fine grain locking to avoid degrading the entire namesystem's throughput.

This message was sent by Atlassian JIRA

View raw message