hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6315) Decouple recording edit logs from FSDirectory
Date Mon, 05 May 2014 22:50:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990064#comment-13990064

Daryn Sharp commented on HDFS-6315:

I am also working towards the goal of removing or minimizing the use of the FSD lock, but
I recall it's being used to protect non-threadsafe data structures (like the inode map and
snapshot manager).  It's spurred by the work to add fine grain locking to the namesystem -
which has been derailed by other pressing issues.  Do keep in mind that hopefully in the next
few months there will not be a globally held FSN so don't entirely remove the FSD lock believing
the FSN lock will cover for it.

bq. The change can be reverted when removing the lock of FSDirectory.

I'm curious what you have in mind.  HDFS-5693 appears to be a valuable change.  I thoughts
deletes used to do something similar while collecting blocks, but that whole region of code
has been changed.

> Decouple recording edit logs from FSDirectory
> ---------------------------------------------
>                 Key: HDFS-6315
>                 URL: https://issues.apache.org/jira/browse/HDFS-6315
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-6315.000.patch, HDFS-6315.001.patch
> Currently both FSNamesystem and FSDirectory record edit logs. This design requires both
FSNamesystem and FSDirectory to be tightly coupled together to implement a durable namespace.
> This jira proposes to separate the responsibility of implementing the namespace and providing
durability with edit logs. Specifically, FSDirectory implements the namespace (which should
have no edit log operations), and FSNamesystem implement durability by recording the edit

This message was sent by Atlassian JIRA

View raw message