Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Tue, 14 Aug 2012 09:38:38 +1100 (NCT)
From: "Daryn Sharp (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <2105942439.4735.1344897518281.JavaMail.jiratomcat@arcas>
In-Reply-To: 
 <538126535.25986.1326215259449.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-7967) Need generalized multi-token
 filesystem support
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433684#comment-13433684 ] 

Daryn Sharp commented on HADOOP-7967:
-------------------------------------

The dilemma over breaking compatibility by reducing the visibility of {{getDelegationToken}} is interesting.  Code that attempts to use this method is incompatible with {{ViewFileSystem}}.  Viewfs can't return tokens and then tasks will unexpectedly fail due to the missing tokens...

I'm willing to revert the visibility, although this is maybe one of the few cases where making things break is the right thing to do?
                
> Need generalized multi-token filesystem support
> -----------------------------------------------
>
>                 Key: HADOOP-7967
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7967
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, security
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HADOOP-7967-2.patch, HADOOP-7967-3.patch, HADOOP-7967-4.patch, HADOOP-7967-compat.patch, hadoop7967-deltas.patch, hadoop7967-javadoc.patch, HADOOP-7967.newapi.2.patch, HADOOP-7967.newapi.3.patch, HADOOP-7967.newapi.patch, HADOOP-7967.patch
>
>
> Multi-token filesystem support and its interactions with the MR {{TokenCache}} is problematic.  The {{TokenCache}} tries to assume it has the knowledge to know if the tokens for a filesystem are available, which it can't possibly know for multi-token filesystems.  Filtered filesystems are also problematic, such as har on viewfs.  When mergeFs is implemented, it too will become a problem with the current implementation.  Currently {{FileSystem}} will leak tokens even when some tokens are already present.
> The decision for token acquisition, and which tokens, should be pushed all the way down into the {{FileSystem}} level.  The {{TokenCache}} should be ignorant and simply request tokens from each {{FileSystem}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira