hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10176) swiftfs doesn't correctly handle object names starting with slash
Date Fri, 03 Jan 2014 09:38:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13861381#comment-13861381
] 

Steve Loughran commented on HADOOP-10176:
-----------------------------------------

well, files with a .crc extension are invisible in the nativeFS (and HDFS?), while HDFS 2.4
will hide subdirs with the .snapshot extension, which is going to force in a migration plan.
Also, I doubt that you can have files called COM1 or LPT1 on the windows native FS due to
[historical reasons|http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx]. 

The S3N filesystem uses {{"_$folder$"}} as its folder separator, so it builds up a very different
model of a directory tree -and accordingly, won't see a lot of paths in an s3 bucket as directories

> swiftfs doesn't correctly handle object names starting with slash
> -----------------------------------------------------------------
>
>                 Key: HADOOP-10176
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10176
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.3.0
>            Reporter: David Dobbins
>            Priority: Minor
>
> When objects are created in swift prefixed by a slash, swiftfs does not correctly expose
the implied directory structure.  For example, given a container with the following objects:
> /foo
> /foo/1
> /foo/2
> teradata
> teradata/part-m
> teradata/part-m-00000
> teradata/part-m-00001
> A GET request against that container will return the list above.  A 'hadoop fs -ls swift://container.service/'
will return the following:
> drwxrwxrwx   -          0 2013-12-19 15:49 /foo
> drwxrwxrwx   -          0 2013-12-19 15:06 /foo/1
> drwxrwxrwx   -          0 2013-12-19 15:09 /foo/2
> drwxrwxrwx   -          0 2013-12-04 04:11 /teradata
> Additionally, if an object named 'foo' is also created, where a GET will return:
> /foo
> /foo/1
> /foo/2
> foo
> rcfile
> teradata
> teradata/part-m
> teradata/part-m-00000
> teradata/part-m-00001
> then 'hadoop fs -ls swift://container.service/' will return the following:
> drwxrwxrwx   -          0 2013-12-19 15:49 /foo
> drwxrwxrwx   -          0 2013-12-19 15:06 /foo/1
> drwxrwxrwx   -          0 2013-12-19 15:09 /foo/2
> drwxrwxrwx   -          0 2013-12-19 19:24 /foo
> drwxrwxrwx   -          0 2013-12-04 04:11 /teradata
> which appears to have a duplicate object "/foo".



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message