hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning
Date Fri, 30 Aug 2019 23:05:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919948#comment-16919948
] 

Arpit Agarwal edited comment on HDFS-14703 at 8/30/19 11:04 PM:
----------------------------------------------------------------

Interesting proposal [~shv] . Thanks for sharing this and the PoC patch. I went through the
doc and the idea seems interesting. I didn't understand how the partitioning scheme works.
Do atomic rename and snapshots still work as before with these changes?

Did you measure write throughput improvement with {{dfs.namenode.edits.asynclogging}}?


was (Author: arpitagarwal):
Interesting proposal [~shv] . Thanks for sharing this and the PoC patch. I went through the
doc and the idea seems interesting. I didn't understand how the partitioning scheme works.
Do atomic rename and snapshots still as before with these changes?

Did you measure write throughput improvement with {{dfs.namenode.edits.asynclogging}}?

> NameNode Fine-Grained Locking via Metadata Partitioning
> -------------------------------------------------------
>
>                 Key: HDFS-14703
>                 URL: https://issues.apache.org/jira/browse/HDFS-14703
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs, namenode
>            Reporter: Konstantin Shvachko
>            Priority: Major
>         Attachments: 001-partitioned-inodeMap-POC.tar.gz, NameNode Fine-Grained Locking.pdf
>
>
> We target to enable fine-grained locking by splitting the in-memory namespace into multiple
partitions each having a separate lock. Intended to improve performance of NameNode write
operations.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message