hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
Date Wed, 06 Feb 2013 17:49:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572611#comment-13572611
] 

Daryn Sharp commented on HADOOP-8545:
-------------------------------------

I can't even hope to review this, but I'll offer some observations:
# Can you instantiate a swift fs with {{Path#getFileSystem(conf)}}?  This is how a fs should
work since it means that existing code will work just by changing the scheme or defaultFS
to swift.
# You probably don't want to store the full uri since when a filesystem is obtained for a
{{Path}}, it's going to be a full URI of the {{Path}}.  Should probably do this: {{this.uri
= URI.create(uri.getScheme()+"://"+uri.getAuthority());}}
# I don't understand the contortions to set the initial working directory.  Maybe {{this.workingDir
= getHomeDirectory()}} like DFS since I think it's doing what you want in a cleaner way.
# The custom {{isDirectory}} behavior is probably better implemented in a {{FileStatus}} subclass.
 Otherwise, {{stat = fs.isDirectory(path)}} != stat.isDirectory()}}
# Seems like there's duplication of {{Path}} and output stream behavior (like the metrics)

The main reason I looked at this is integration with security.  You seem to have your own
token implementation, but it's not tied into {{getDelegationToken}}.  If you don't implement
that, I don't think jobs are going to be able to use this fs.

                
> Filesystem Implementation for OpenStack Swift
> ---------------------------------------------
>
>                 Key: HADOOP-8545
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8545
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 2.0.3-alpha, 1.1.2
>            Reporter: Tim Miller
>            Assignee: Dmitry Mezhensky
>         Attachments: HADOOP-8545-1.patch, HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch,
HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-javaclouds-2.patch,
HADOOP-8545.patch, HADOOP-8545.patch
>
>
> Add a filesystem implementation for OpenStack Swift object store, similar to the one
which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message