hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2052) FSNamesystem should not sync the log with the write lock held
Date Tue, 14 Jun 2011 07:28:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049028#comment-13049028
] 

Suresh Srinivas commented on HDFS-2052:
---------------------------------------

Eli, the *Internal variant of method should not sync editlogs. It should be done from the
variant of the method that calls the *Internal method. Will that not solve the problem?

BTW I also think, in startFileInternal(), if we delete a file to overwrite and later we fail
to create the file, we should resurrect the file?

> FSNamesystem should not sync the log with the write lock held
> -------------------------------------------------------------
>
>                 Key: HDFS-2052
>                 URL: https://issues.apache.org/jira/browse/HDFS-2052
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Eli Collins
>
> FSNamesystem#deleteInternal releases the write lock before syncing the log, however FSNamesystem#startFileInternal
calls delete -> deleteInternal with the write lock held, which means deleteInternal will
sync the log while holding the lock. We could fix cases like this by passing  a flag indicating
whether the function should sysnc (eg in this case the sysnc is not necessary because startFileInternals
callers will sync the log) or modify the current calls to sync to flag that a sync is necessary
before returning to the caller rather than doing the sync right at the call sight. This way
the cost of syncing the log could be amortized over multiple function calls (and potentially
multiple RPCs if we didn't mind introducing some synchronization).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message