hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14716) SwiftNativeFileSystem should not eat the exception when rename
Date Wed, 02 Aug 2017 09:20:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110607#comment-16110607

Steve Loughran commented on HADOOP-14716:

This is actually a fundamental issue with rename() itself, many failures (source doesn't exist,
destination is a file) are required to be caught and downgraded to a "return false", which
hides many problems and means that application code often goes 

if (!fs.rename(src, dest)) throw new IOException("rename failed")

# there's a protected rename operation,  {{FileSystem.rename(final Path src, final Path dst,
final Rename... options)}}  which I've proposed making public and adopting broadly for renaming,
which means "more spec, tests, update uses". See HADOOP-11452 for details. I've not done anything
on that since january
# we ought to split out "exceptions to swallow" from "exceptions to throw up": authentication,
networking, &c should be thrown. If the Swift client is catching them too, it shouldn't.
Patches welcome
# And yes, logging too

Take a look at [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L690]
to see how rename failures are handled there: the {{innerRename()}} method does throw exceptions,
including explicit ones with conditions not being met; the outer one catches and downgrades

> SwiftNativeFileSystem should not eat the exception when rename
> --------------------------------------------------------------
>                 Key: HADOOP-14716
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14716
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 2.8.1, 3.0.0-alpha3
>            Reporter: Chen He
>            Assignee: Chen He
>            Priority: Minor
> Currently, if "rename" will eat excpetions and return "false" in SwiftNativeFileSystem.
It is not easy for user to find root cause about why rename failed. It has to, at least, write
out some logs instead of directly eats these exceptions.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message