hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4258) Rename of Being Written Files
Date Wed, 06 Feb 2013 21:19:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572842#comment-13572842

Aaron T. Myers commented on HDFS-4258:

I would still prefer this work be moved to a branch.

I'd also like to see if we can address the motivation for this issue (rename of being written
files) without introducing an INode ID like this. It may well be that introducing a unique
INode ID as has been described here is a good architectural change, but I suspect it's not
actually necessary to address the bug described by this JIRA. I say this mostly because it
seems this JIRA has already concluded that the appropriate thing to do is to error out if
a file which is actively being written is renamed, as mentioned in this comment:

2. add new addBlock interface which takes the id as an additional field. The id is checked
by checkLease(). If it doesn't match current id in the found inode, NN fails the request.

Wouldn't checking the path in the lease along with the clientName be sufficient to detect
that a being-written file had been renamed, and thus the NN should fail the request?
> Rename of Being Written Files
> -----------------------------
>                 Key: HDFS-4258
>                 URL: https://issues.apache.org/jira/browse/HDFS-4258
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client, namenode
>    Affects Versions: 3.0.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Brandon Li
>         Attachments: HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch
> When a being written file or it's ancestor directories is renamed, the path in the file
lease is also renamed.  Then the writer of the file usually will fail since the file path
in the writer is not updated.
> Moreover, I think there is a bug as follow:
> # Client writes 0's to F_0="/foo/file" and writes 1's to F_1="/bar/file" at the same
> # Rename /bar to /baz
> # Rename /foo to /bar
> Then, writing to F_0 will fail since /foo/file does not exist anymore but writing to
F_1 may succeed since /bar/file exits as a different file.  In such case, the content of /bar/file
could be partly 0's and partly 1's.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message