hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4043) Secret keys set in Credentials are not seen by tasks
Date Thu, 22 Mar 2012 21:30:24 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe updated MAPREDUCE-4043:

    Attachment: MAPREDUCE-4043.patch

I don't believe MAPREDUCE-3727 is relevant as the property being unset in that change needs
to be set by the job submitter and that's not happening in this case.

I believe the keys are dropped by tasks because the TaskImpl/TaskAttemptImpl constructors
take a collection of tokens instead of credentials.  Therefore when the map and reduce tasks
attempts are constructed by the AM, it ends up dropping the keys out of the credentials since
there's only parameters to pass tokens.

Here's a patch that changes the collection of tokens argument to credentials, which contain
tokens and secret keys.  Manual testing with a toy job that sends a secret key works.  Needs
unit tests.
> Secret keys set in Credentials are not seen by tasks
> ----------------------------------------------------
>                 Key: MAPREDUCE-4043
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4043
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2, security
>    Affects Versions: 0.23.2
>            Reporter: Jason Lowe
>            Priority: Blocker
>         Attachments: MAPREDUCE-4043.patch
> The following scenario works in 0.20.205 but no longer works in 0.23:
> 1) During job submission, a secret key is set by calling jobConf.getCredentials().addSecretKey(Text,
> 2) A map task retrieves the secret key by calling jobConf.getCredentials().getSecretKey(Text)
> In 205 the secret key is retrieved successfully but in 0.23 the secret key is missing.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message