hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7933) Viewfs changes for MAPREDUCE-3529
Date Tue, 20 Dec 2011 22:45:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173580#comment-13173580
] 

Hadoop QA commented on HADOOP-7933:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12508137/HADOOP-7933.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    -1 javadoc.  The javadoc tool appears to have generated 5 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these unit tests:
                  org.apache.hadoop.fs.TestFilterFileSystem

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/486//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/486//console

This message is automatically generated.
                
> Viewfs changes for MAPREDUCE-3529
> ---------------------------------
>
>                 Key: HADOOP-7933
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7933
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.23.0
>            Reporter: Siddharth Seth
>            Priority: Critical
>         Attachments: HADOOP-7933.txt, HDFS2665_v1.txt, HDFS2665_v1.txt
>
>
> ViewFs.getDelegationTokens returns a list of tokens for the associated namenodes. Credentials
serializes these tokens using the service name for the actual namenodes. Effectively, tokens
are not cached for viewfs (some more details in MR 3529). Affects any job which uses the TokenCache
in tasks along with viewfs (some Pig jobs).
> Talk to Jitendra about this, some options
> 1. Change Credentials.getAllTokens to return the key, instead of just a token list (associate
the viewfs canonical name with a token in credentials)
> 2. Have viewfs issue a fake token.
> Both of these would allow for a single viewfs configuration only.
> 3. An additional API in FileSystem - something like getDelegationTokens(String renewer,
Credentials credentials) - which would check the credentials object before making token requests
to the actual namenode.
> 4. An additional API in FileSystem - getCanonicalServiceNames - similar to getDelegationTokens,
which would return service names for the actual namenodes. TokenCache/Credentials can work
using this list.
> 5. have getDelegationTokens check the current UGI - and fetch tokens only if they don't
exist.
> Have a quick patch for 3, along with associated MR changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message