hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3825) Need generalized multi-token filesystem support
Date Fri, 10 Feb 2012 23:01:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205833#comment-13205833

Sanjay Radia commented on MAPREDUCE-3825:

Here is how Solution 2 will be used and implemented.

// Credentials is a map<serviceName, Token>
void FileSystem#addDelegationTokens(renewer, credentials); // NewAPi for FileSystem
// note the old #getDelegationTokens(...) methods in FileSystem are no longer needed.

// A Useful Utility - so that the TokenCache in MR can be easily implemented
FileUtil:GetTokens(renewer, path[] ps, credentials) {
  foreach (p in ps) {
    GetFileSystem(p).addDelegationTokens(renwer, credentials);

// Two implementation examples - viewfs and DistributedFileSystem

ViewFileSystem#addDelegationTokens(renewer, credentials) {// contains embedded FSs as mounts
 foreach (mountFs in mountPoints) {
    mountFs.addDelegationTokens(renewer, credentials);

DistributedFileSystem#addDelegationTokens(renewer, credentials) { // a leaf file system.
  // I am ignoring the race condition across contains() and add();
  myServiceName = getCanonicalServiceName();
  if (credentials.contains(myServiceName) {
  myDelegationToken = getDTfromMyNN();
  credentials.add(myServiceName, myDelegationToken);

> Need generalized multi-token filesystem support
> -----------------------------------------------
>                 Key: MAPREDUCE-3825
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: MAPREDUCE-3825.patch, TokenCache.pdf
> This is the counterpart to HADOOP-7967.  The token cache currently tries to assume a
filesystem's token service key.  The assumption generally worked while there was a one to
one mapping of filesystem to token.  With the advent of multi-token filesystems like viewfs,
the token cache will try to use a service key (ie. for viewfs) that will never exist (because
it really gets the mounted fs tokens).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message