hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2551) fuse-dfs's close() is asynchronous
Date Wed, 03 Oct 2012 21:23:48 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468847#comment-13468847
] 

Colin Patrick McCabe commented on HDFS-2551:
--------------------------------------------

To add more detail here:

Every time you call close() on a FUSE file descriptor, the FUSE flush() method is called synchronously.
 That method calls hsync(), which will make the file visible in the HDFS namespace.

When there are no more file descriptors left referring to the open file, it is put in a queue,
so that the FUSE release() function can be run asychronously at some later time.

So if you have code like this:
{code}
fd = open(NAME, ...)
write(fd, ...)
close(fd)
fd2 = open(NAME, ...)
{code}

Chances are that the second open will fail because the underlying HDFS file is still open.

One way to fix this would be to call hdfsClose in the fuse flush() method.  Then, if the file
still needed to be written to after that, we'd have to re-open it using append.  This would
also have the beneficial effect of removing an unecessary call to hdfsFlush in the common
case.
                
> fuse-dfs's close() is asynchronous
> ----------------------------------
>
>                 Key: HDFS-2551
>                 URL: https://issues.apache.org/jira/browse/HDFS-2551
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: fuse-dfs
>    Affects Versions: 0.20.205.0, 0.23.0
>            Reporter: Todd Lipcon
>
> FUSE's "release" operation is called asynchronously when a file is closed from userspace.
This means that, if a program writes a file, closes it, and immediately re-opens it, it might
not succeed. We should figure out some workaround or trick to make close synchronous, if possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message