hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5375) Delegation Token renewal exception in jobtracker logs
Date Fri, 05 Jul 2013 03:19:48 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated MAPREDUCE-5375:
-------------------------------------

       Resolution: Fixed
    Fix Version/s: 1.2.1
           Status: Resolved  (was: Patch Available)

I just committed this. Thanks Venkat, Vinod & Sid!
                
> Delegation Token renewal exception in jobtracker logs
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-5375
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5375
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 1.2.0
>            Reporter: Venkat Ranganathan
>            Assignee: Venkat Ranganathan
>            Priority: Critical
>             Fix For: 1.2.1
>
>         Attachments: MRDelegationIssue-MR-5375.patch
>
>
> Filing on behalf of [~venkatnrangan] who found this originally and provided a patch.
> Saw this in the JT logs while oozie tests were running with Hadoop.
> When Oozie java action is executed, the following shows up in the job tracker log.
> {code}
> ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: Exception renewing
tokenIdent: 00 07 68 64 70 75 73 65 72 06 6d 61 70 72 65 64 26 6f 6f 7a 69 65 2f 63 6f 6e
64 6f 72 2d 73 65 63 2e 76 65 6e 6b 61 74 2e 6f 72 67 40 76 65 6e 6b 61 74 2e 6f 72 67 8a
01 3e a6 87 5e 5b 8a 01 3e ca 93 e2 5b 02 02, Kind: MAPREDUCE_DELEGATION_TOKEN, Service: ip:50300.
Not rescheduled
> org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException:
Client jt/host@domain.com tries to renew a token with renewer specified as mapred
>         at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
>         at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:3878)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
>         at org.apache.hadoop.ipc.Client.call(Client.java:1118)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>         at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>         at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
>         at org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:578)
>         at org.apache.hadoop.security.token.Token.renew(Token.java:309)
>         at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
>         at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>         at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> {code}
> Setting the renewer to Kerberos Local name does not help because AbstractDelegationTokenIdentifier
sets the renewer to Kerberos shortname but JobTracker.renewDelegationToken uses the fullName.
 This essentially causes the renewal to fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message