hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3825) Need generalized multi-token filesystem support
Date Fri, 10 Feb 2012 23:01:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205833#comment-13205833
] 

Sanjay Radia commented on MAPREDUCE-3825:
-----------------------------------------

Here is how Solution 2 will be used and implemented.

{code}
// Credentials is a map<serviceName, Token>
void FileSystem#addDelegationTokens(renewer, credentials); // NewAPi for FileSystem
// note the old #getDelegationTokens(...) methods in FileSystem are no longer needed.


// A Useful Utility - so that the TokenCache in MR can be easily implemented
FileUtil:GetTokens(renewer, path[] ps, credentials) {
  foreach (p in ps) {
    GetFileSystem(p).addDelegationTokens(renwer, credentials);
  return;
}

// Two implementation examples - viewfs and DistributedFileSystem

ViewFileSystem#addDelegationTokens(renewer, credentials) {// contains embedded FSs as mounts
 foreach (mountFs in mountPoints) {
    mountFs.addDelegationTokens(renewer, credentials);
  }
  return;
}


DistributedFileSystem#addDelegationTokens(renewer, credentials) { // a leaf file system.
  // I am ignoring the race condition across contains() and add();
  myServiceName = getCanonicalServiceName();
  if (credentials.contains(myServiceName) {
    return;
  }
  myDelegationToken = getDTfromMyNN();
  credentials.add(myServiceName, myDelegationToken);
  return;
}

{code}
                
> Need generalized multi-token filesystem support
> -----------------------------------------------
>
>                 Key: MAPREDUCE-3825
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3825
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: MAPREDUCE-3825.patch, TokenCache.pdf
>
>
> This is the counterpart to HADOOP-7967.  The token cache currently tries to assume a
filesystem's token service key.  The assumption generally worked while there was a one to
one mapping of filesystem to token.  With the advent of multi-token filesystems like viewfs,
the token cache will try to use a service key (ie. for viewfs) that will never exist (because
it really gets the mounted fs tokens).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message