hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14745) s3a getFileStatus can't return expect result when existing a file and directory with the same name
Date Tue, 08 Aug 2017 13:10:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118315#comment-16118315
] 

Steve Loughran commented on HADOOP-14745:
-----------------------------------------

Sorry, no. it is possible to create s3 data through other tooling with names s3a can't handle

* data in path/
* data in path////path2
* +probably some characters in path elements we don't allow

Not only that, s3a can and will delete what it thinks are mock directories, e.g /path/, without
even checking to see if they contain data. 

You're going to have to set up a process which doesn't create paths which confuse S3a. Hadoop
s3a makes sure it does this itself. For other tools, you are going to have to be aware of
the limitations and avoid them.

Sorry. Closing as a WONTFIX, for reasons discussed.

> s3a getFileStatus can't return expect result when existing a file and directory with
the same name
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14745
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14745
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Yonger
>            Assignee: Yonger
>
> {code}
> [**** hadoop-aws]# /root/hadoop/s3cmd/s3cmd ls s3://test-aws-s3a/user/root/
>                        DIR   s3://test-aws-s3a/user/root/ccc/
> 2017-08-08 07:04         0   s3://test-aws-s3a/user/root/ccc
> {code}
> if we expect to ccc is a directory by code :
> {code}
>     Path test=new Path("ccc");
>     fs.getFileStatus(test);
> {code}
> actually, it will tell us it is a file:
> {code}
> 2017-08-08 15:08:40,566 [JUnit-case1] DEBUG s3a.S3AFileSystem (S3AFileSystem.java:getFileStatus(1576))
- Getting path status for s3a://test-aws-s3a/user/root/ccc  (user/root/ccc)
> 2017-08-08 15:08:40,566 [JUnit-case1] DEBUG s3a.S3AFileSystem (S3AStorageStatistics.java:incrementCounter(60))
- object_metadata_requests += 1  ->  3
> 2017-08-08 15:08:40,580 [JUnit-case1] DEBUG s3a.S3AFileSystem (S3AFileSystem.java:getFileStatus(1585))
- Found exact file: normal file
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message