Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Thu, 7 Feb 2013 07:33:14 +0000 (UTC)
From: "Dmitry Mezhensky (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12596452.1341003756772.259024.1360222394817@arcas>
In-Reply-To: <JIRA.12596452.1341003756772@arcas>
References: <JIRA.12596452.1341003756772@arcas>
Subject: [jira] [Commented] (HADOOP-8545) Filesystem Implementation for
 OpenStack Swift
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573274#comment-13573274 ] 

Dmitry Mezhensky commented on HADOOP-8545:
------------------------------------------

|1. Can you instantiate a swift fs with Path#getFileSystem(conf)? This is how a fs should work since it means that existing code will work just by changing the scheme or defaultFS to swift.

It instantiates correctly if required configuration present 

| 2. You probably don't want to store the full uri since when a filesystem is obtained for a Path, it's going to be a full URI of the Path. Should probably do this: this.uri = URI.create(uri.getScheme()+"://"+uri.getAuthority());

I need full url for different containers and swift installations support, for example different containers: swift://data.rackspace/var/runs and swift://result.rackspace/home/hadoop (data and result are different containers in Swift)

| 3. I don't understand the contortions to set the initial working directory. Maybe this.workingDir = getHomeDirectory() like DFS since I think it's doing what you want in a cleaner way.

I'll try to simplify it

| 4. The custom isDirectory behavior is probably better implemented in a FileStatus subclass. Otherwise, stat = fs.isDirectory(path) != stat.isDirectory()}}

Directory support in Swift is quite complicated, I think we can't do so.

| The main reason I looked at this is integration with security. You seem to have your own token implementation, but it's not tied into getDelegationToken. If you don't implement that, I don't think jobs are going to be able to use this fs.

Yes, Swift has it's ouw token implementation.
                
> Filesystem Implementation for OpenStack Swift
> ---------------------------------------------
>
>                 Key: HADOOP-8545
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8545
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 2.0.3-alpha, 1.1.2
>            Reporter: Tim Miller
>            Assignee: Dmitry Mezhensky
>         Attachments: HADOOP-8545-1.patch, HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, HADOOP-8545.patch
>
>
> Add a filesystem implementation for OpenStack Swift object store, similar to the one which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira