hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4006) YARN ATS Alternate Kerberos HTTP Authentication Changes
Date Mon, 02 May 2016 21:29:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267539#comment-15267539
] 

Allen Wittenauer edited comment on YARN-4006 at 5/2/16 9:28 PM:
----------------------------------------------------------------

I suspect the "Hadoop Alternate Kerberos classes" referenced above are, in fact, HADOOP-9054.


I'm sitting here looking what I suspect are the exact same failures that [~gss2002] saw. 
Stack trace on job launch is:

{code}
java.lang.NullPointerException
    at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.getDelegationToken(DelegationTokenAuthenticator.java:170)
    at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.getDelegationToken(DelegationTokenAuthenticatedURL.java:371)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:359)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:351)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$5.run(TimelineClientImpl.java:451)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:183)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.operateDelegationToken(TimelineClientImpl.java:466)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:363)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:348)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:329)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:249)
    at org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:290)
    at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:290)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
    at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:273)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:130)
    at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:138)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}

... attempting to use this patch does not fix the issue, however. :(




was (Author: aw):
I suspect the "Hadoop Alternate Kerberos classes" referenced above are, in fact, HADOOP-9054.


I'm sitting here looking what I suspect are the exact same failures that [~gss2002] saw. 
Stack trace on job launch is:

{code}
java.lang.NullPointerException
    at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.getDelegationToken(DelegationTokenAuthenticator.java:170)
    at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.getDelegationToken(DelegationTokenAuthenticatedURL.java:371)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:359)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:351)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$5.run(TimelineClientImpl.java:451)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:183)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.operateDelegationToken(TimelineClientImpl.java:466)
    at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:363)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:348)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:329)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:249)
    at org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:290)
    at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:290)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
    at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:273)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:130)
    at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:138)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}

> YARN ATS Alternate Kerberos HTTP Authentication Changes
> -------------------------------------------------------
>
>                 Key: YARN-4006
>                 URL: https://issues.apache.org/jira/browse/YARN-4006
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: security, timelineserver
>    Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.5.1, 2.6.1, 2.8.0, 2.7.1, 2.7.2
>            Reporter: Greg Senia
>            Assignee: Greg Senia
>         Attachments: YARN-4006-branch-trunk.patch, YARN-4006-branch2.6.0.patch
>
>
> When attempting to use The Hadoop Alternate Authentication Classes. They do not exactly
work with what was built with YARN-1935.
> I went ahead and made the following changes to support using a Custom AltKerberos DelegationToken
custom class.
> Changes to: TimelineAuthenticationFilterInitializer.class
> {code}
>    String authType = filterConfig.get(AuthenticationFilter.AUTH_TYPE);
>     LOG.info("AuthType Configured: "+authType);
>     if (authType.equals(PseudoAuthenticationHandler.TYPE)) {
>       filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           PseudoDelegationTokenAuthenticationHandler.class.getName());
>         LOG.info("AuthType: PseudoDelegationTokenAuthenticationHandler");
>     } else if (authType.equals(KerberosAuthenticationHandler.TYPE) || (UserGroupInformation.isSecurityEnabled()
&& conf.get("hadoop.security.authentication").equals(KerberosAuthenticationHandler.TYPE)))
{
>       if (!(authType.equals(KerberosAuthenticationHandler.TYPE))) {
>         filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           authType);
>         LOG.info("AuthType: "+authType);
>       } else {
>         filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           KerberosDelegationTokenAuthenticationHandler.class.getName());
>         LOG.info("AuthType: KerberosDelegationTokenAuthenticationHandler");
>       } 
>       // Resolve _HOST into bind address
>       String bindAddress = conf.get(HttpServer2.BIND_ADDRESS);
>       String principal =
>           filterConfig.get(KerberosAuthenticationHandler.PRINCIPAL);
>       if (principal != null) {
>         try {
>           principal = SecurityUtil.getServerPrincipal(principal, bindAddress);
>         } catch (IOException ex) {
>           throw new RuntimeException(
>               "Could not resolve Kerberos principal name: " + ex.toString(), ex);
>         }
>         filterConfig.put(KerberosAuthenticationHandler.PRINCIPAL,
>             principal);
>       }
>     }
>  {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message