hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6193) HftpFileSystem open should throw FileNotFoundException for non-existing paths
Date Sun, 06 Apr 2014 19:06:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13961490#comment-13961490
] 

Steve Loughran commented on HDFS-6193:
--------------------------------------

linking to HADOOP-9361 and FS semantics.

Failing on the open if a file is not found is a core expectation of filesystems.

We could optimise any of the web filesystems by not doing that open (e,g, S3, s3n, swift)
and waiting for the first seek. But we don't because things expect missing files to not be
there.

Interesting that FileSystemContractBaseTest doesn't catch this

> HftpFileSystem open should throw FileNotFoundException for non-existing paths
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-6193
>                 URL: https://issues.apache.org/jira/browse/HDFS-6193
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.3.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>            Priority: Blocker
>
> WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles non-existing paths.

> - 'open', does not really open anything, i.e., it does not contact the server, and therefore
cannot discover FileNotFound, it's deferred until next read. It's counterintuitive and not
how local FS or HDFS work. In POSIX you get ENOENT on open. [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST instead of SC_NOT_FOUND
for non-exitsing paths



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message