hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11452) Revisit FileSystem.rename(path, path, options)
Date Wed, 04 Jan 2017 15:52:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798554#comment-15798554

Steve Loughran commented on HADOOP-11452:

also, no explicit check in the base rename of {{rename("/dir1", "/dir1/dir2",OVERWRITE)}}.
If dir2 doesn't exist, call is handled down and relies on base rename/2 to get it right. If
dir2 does exist, there's a check for dir2 not being empty.

Seems to me there should be a few extra precondition checks on fully qualified paths. Note
that {{DelegateToFileSystem}} rejects unqualified paths, so there's no need for checks in
rename/3. If made public, the qualification would have to go in.

# src != dest else raise an exception. Or, if this is deemed to be a no-op, exit immediately.
# dest is not under src.
# src is not under dest. This implicitly fails right now as dest would be a non-empty directory.
But it would be good to call out.

The implication here is that things are more complex than just making the method public. It'll
need a full contract test suite, review of what HDFS really does, fixes for what appear to
be some major defects —which has implications for FileContext too.

> Revisit FileSystem.rename(path, path, options)
> ----------------------------------------------
>                 Key: HADOOP-11452
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11452
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: fs
>    Affects Versions: 2.7.3
>            Reporter: Yi Liu
>            Assignee: Steve Loughran
> Currently in {{FileSystem}}, {{rename}} with _Rename options_ is protected and with _deprecated_
annotation. And the default implementation is not atomic.
> So this method is not able to be used outside. On the other hand, HDFS has a good and
atomic implementation. (Also an interesting thing in {{DFSClient}}, the _deprecated_ annotations
for these two methods are opposite).
> It makes sense to make public for {{rename}} with _Rename options_, since it's atomic
for rename+overwrite, also it saves RPC calls if user desires rename+overwrite.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message