hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4258) Rename of Being Written Files
Date Thu, 07 Feb 2013 04:49:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573185#comment-13573185
] 

Aaron T. Myers commented on HDFS-4258:
--------------------------------------

Thanks a lot for the pointer to the test, Brandon. Two things:

# It looks to me like that test doesn't actually exercise the scenario described in the description
of this JIRA, since the test doesn't write to the file before the rename. It creates a file,
then a parent of that empty file is renamed, then a new empty file is created with the same
name, and then the first client writes to the recreated file. I suppose we can consider that
a bug as well, since the first FSOutputStream can now still write to the empty file, but it's
not nearly so bad as was first described by this JIRA, since we're not ending up with data
in the file from two different clients.
# I think Daryn makes a really good point - to address this issue of clients still writing
to renamed files, can't we just revoke the relevant leases at rename time? That seems like
a way simpler way to fix this bug than adding an INode ID and threading that through a bunch
of RPCs in the ClientProtocol, and has the other advantage Daryn mentioned.
                
> Rename of Being Written Files
> -----------------------------
>
>                 Key: HDFS-4258
>                 URL: https://issues.apache.org/jira/browse/HDFS-4258
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client, namenode
>    Affects Versions: 3.0.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Brandon Li
>         Attachments: HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch
>
>
> When a being written file or it's ancestor directories is renamed, the path in the file
lease is also renamed.  Then the writer of the file usually will fail since the file path
in the writer is not updated.
> Moreover, I think there is a bug as follow:
> # Client writes 0's to F_0="/foo/file" and writes 1's to F_1="/bar/file" at the same
time.
> # Rename /bar to /baz
> # Rename /foo to /bar
> Then, writing to F_0 will fail since /foo/file does not exist anymore but writing to
F_1 may succeed since /bar/file exits as a different file.  In such case, the content of /bar/file
could be partly 0's and partly 1's.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message