hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4258) Rename of Being Written Files
Date Mon, 11 Feb 2013 15:59:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13575857#comment-13575857
] 

Daryn Sharp commented on HDFS-4258:
-----------------------------------

I think FD-like behavior is a good idea, although we'd have to carefully test that MR and
other components aren't intentionally or unintentionally relying on renaming a file breaking
a lease.  Or if something is writing a log and assumes a broken write due to the file missing
means that log rotation has occurred.  In both cases, renamed files that were assumed to become
immutable will now continue to mutate.

We'll also have to carefully ensure that security cannot be bypassed by knowing or guessing
fileIds.

In the meantime, can we implement HDFS-4437?  We've been running into issues with daemons
holding leases to renamed files.
                
> Rename of Being Written Files
> -----------------------------
>
>                 Key: HDFS-4258
>                 URL: https://issues.apache.org/jira/browse/HDFS-4258
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client, namenode
>    Affects Versions: 3.0.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Brandon Li
>         Attachments: HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch
>
>
> When a being written file or it's ancestor directories is renamed, the path in the file
lease is also renamed.  Then the writer of the file usually will fail since the file path
in the writer is not updated.
> Moreover, I think there is a bug as follow:
> # Client writes 0's to F_0="/foo/file" and writes 1's to F_1="/bar/file" at the same
time.
> # Rename /bar to /baz
> # Rename /foo to /bar
> Then, writing to F_0 will fail since /foo/file does not exist anymore but writing to
F_1 may succeed since /bar/file exits as a different file.  In such case, the content of /bar/file
could be partly 0's and partly 1's.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message