hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3825) Need generalized multi-token filesystem support
Date Fri, 10 Feb 2012 23:48:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205882#comment-13205882

Daryn Sharp commented on MAPREDUCE-3825:

I think requiring every filesystem to bracket its token retrieval with identical "check for
my token" and "set my token" is brittle.  It's an invasive change that isn't backwards compatible,
so any filesystem that doesn't properly do a copy-n-paste will cause duplicate tokens.  If
we want to universally change the behavior, we have to change the filesystems again.

I feel it's much safer for a filesystem to implement primitives that a common method uses.
 My proposed FileSystem#getDelegationTokens does just that.  All a filesystem has to do is
implement getDelegationToken & maybe getFileSystems if it has multiple tokens.  Everything
else is managed for the filesystem.  I'd like to make FileSystem#getDelegationsTokens a final
method to enforce the consistency and prevent any filesystem from trying to directly manipulate
the credentials.  If we want to change the implementation in the future, there's only one
place, in our control, that needs to be changed.

The sample code also prevents viewfs from shorting out on calls to the same filesystem.  It
can't be solved by uniquing the fs list.  Otherwise, it's a repeat of the TokenCache 1-to-1
mapping of service to a specific token problem.  We can't avoid this by uniquing the fs list
in viewfs because the underlying mounts might have multiple filesystems, or it might be returning
a null service (filtered) yet have a contained filesystem with a token.

Once MAPREDUCE-3849 is incorporated, I can fix TokenCache to eliminate the 1-to-1 mapping
problem by simply calling getDelegationTokens on the filesystems.
> Need generalized multi-token filesystem support
> -----------------------------------------------
>                 Key: MAPREDUCE-3825
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: MAPREDUCE-3825.patch, TokenCache.pdf
> This is the counterpart to HADOOP-7967.  The token cache currently tries to assume a
filesystem's token service key.  The assumption generally worked while there was a one to
one mapping of filesystem to token.  With the advent of multi-token filesystems like viewfs,
the token cache will try to use a service key (ie. for viewfs) that will never exist (because
it really gets the mounted fs tokens).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message