hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11452) Revisit FileSystem.rename(path, path, options)
Date Wed, 04 Jan 2017 11:45:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798027#comment-15798027

Steve Loughran commented on HADOOP-11452:

In HADOOP-13856 I'd proposed a simpler action: make rename/3 public, specify it and that it
SHOULD be atomic, add tests for correct precond/postcond

Even if the base implementation is not atomic, we can make use of the fact that it is stricter
in checks and failure reporting than rename/2, so is more consistent to use. Subclasses of
FS can implement more efficient and/or atomic versions of the API call, something HDFS already
does. For S3A we can't do atomic without s3guard, but we can at least coalesce the checks
to have a faster rename sequence than rename/3, and that better reporting. Currently in rename/2
catches some exceptions and downgrades to a true/false return code, whereas for rename/3 it
can throw them all the way up.

> Revisit FileSystem.rename(path, path, options)
> ----------------------------------------------
>                 Key: HADOOP-11452
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11452
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: fs
>    Affects Versions: 2.7.3
>            Reporter: Yi Liu
>            Assignee: Steve Loughran
> Currently in {{FileSystem}}, {{rename}} with _Rename options_ is protected and with _deprecated_
annotation. And the default implementation is not atomic.
> So this method is not able to be used outside. On the other hand, HDFS has a good and
atomic implementation. (Also an interesting thing in {{DFSClient}}, the _deprecated_ annotations
for these two methods are opposite).
> It makes sense to make public for {{rename}} with _Rename options_, since it's atomic
for rename+overwrite, also it saves RPC calls if user desires rename+overwrite.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message