hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running clients which are not kerberos authenticated
Date Thu, 10 Nov 2011 18:46:52 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated MAPREDUCE-3380:
-----------------------------------------------

    Affects Version/s:     (was: 0.24.0)
        Fix Version/s: 0.23.1
              Summary: Token infrastructure for running clients which are not kerberos authenticated
 (was: JobClient.getDelegationToken() returns NULL)

This is a deep issue. I knew about this in the back of my mind, but put it on the back-burner
because of more pressing tasks.

Client (the usual kerberos authenticated one) talks to RM (for submitting apps and getting
application-status), gets a client-token which it then uses to connect to the AM, and the
finally connects to the JobHistoryServer to obtain the status when the job completes.

To make oozie work (i.e. making a client that isn't kerberos authenticated), we need the following:
 - To communicate to the RM, RM needs to support ApplicationDelegationTokens which the JobClient
should then propagate to the Oozie action.
 - To communicate to the AM, the present day client-token generated and given to the client
by the RM should suffice.
 - To communicate to the JobHistory server, again we need some delegation token. I guess we
can use the mapreduce JobToken itself but AM somehow needs to let the JHS know about it -
through JobHistory file, or a direct communication protocol.

We should perhaps split it into two separate tickets.

Modifying title to reflect the issues here.

I really am not sure about the timelines for fixing all this, but setting 0.23.1 tentatively.
                
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3380
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mr-am
>    Affects Versions: 0.23.0
>            Reporter: Alejandro Abdelnur
>            Priority: Blocker
>             Fix For: 0.23.1
>
>         Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when
trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken()
that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken().
And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message