hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4006) YARN ATS Alternate Kerberos HTTP Authentication Changes
Date Sat, 21 May 2016 13:48:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295033#comment-15295033

Allen Wittenauer commented on YARN-4006:

bq. Manual testing is acceptable if it is deemed that this is not testable via unit-tests.
I don't see any comments from anyone saying that.

Then I'll say it.  It's pretty clear just looking at TestTimelineAuthenticationFilterInitializer.java
that it doesn't test nearly enough because of the difficulty involved.  There are *zero* Kerberos
tests, for example. Whoever committed clearly understood the difficulty invovled, otherwise
it wouldn't have made it in right? ;)

bq.  we still don't completely understand the patch 

The yarn client expects to be able to ask the auth handler to get a delegation token over
REST if security is enabled.  The problem, however, is that if AltKerberos is enabled, there's
no way for the AltKerberos implementation to call the DelegationToken manager because the
security check is built around the idea that there are only two auth handlers (none or Kerberos).
 So yarn job submission fails because the token fetch fails on the server side because the
handler in use isn't either of those.  

It's pretty obviously broken if one sets up an AltKerb handler and actually tries to use it.

The patch here *does* require a change to AltKerberos implementations (they'll likely need
to super() up to call the token management routines), but there's really no other choice.
It could be easily argued that the AltKerberos interface/design specs were incomplete and
this forces that hole closed.  There's also the whole side-thing around timeline server not
really being deployed at a lot of sites (for various reasons) due to being optional.  Hadoop
2.x has introduced significantly worse incompatibilities throughout it's lifecycle, but at
least this one is realistic and understandable.

It's worth pointing out that this is probably the first time that I'm aware of that REST is
being used instead of RPC inside Java code in Hadoop. It's not, however, the first time that
AltKerberos was completely broken because authors of code didn't realize that there were was
more than one handler.  The HDFS browser fiasco effectively killed 2.5 and 2.6 upgrades for
the exact same reason.  (although, with 2.6, nothing of value was lost.) Now that both HDFS
and YARN have broken something in this space, let's hope there isn't a threepeat.

> YARN ATS Alternate Kerberos HTTP Authentication Changes
> -------------------------------------------------------
>                 Key: YARN-4006
>                 URL: https://issues.apache.org/jira/browse/YARN-4006
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: security, timelineserver
>    Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.5.1, 2.6.1, 2.8.0, 2.7.1, 2.7.2
>            Reporter: Greg Senia
>            Assignee: Greg Senia
>            Priority: Blocker
>         Attachments: YARN-4006-branch-trunk.patch, YARN-4006-branch2.6.0.patch, sample-ats-alt-auth.patch
> When attempting to use The Hadoop Alternate Authentication Classes. They do not exactly
work with what was built with YARN-1935.
> I went ahead and made the following changes to support using a Custom AltKerberos DelegationToken
custom class.
> Changes to: TimelineAuthenticationFilterInitializer.class
> {code}
>    String authType = filterConfig.get(AuthenticationFilter.AUTH_TYPE);
>     LOG.info("AuthType Configured: "+authType);
>     if (authType.equals(PseudoAuthenticationHandler.TYPE)) {
>       filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           PseudoDelegationTokenAuthenticationHandler.class.getName());
>         LOG.info("AuthType: PseudoDelegationTokenAuthenticationHandler");
>     } else if (authType.equals(KerberosAuthenticationHandler.TYPE) || (UserGroupInformation.isSecurityEnabled()
&& conf.get("hadoop.security.authentication").equals(KerberosAuthenticationHandler.TYPE)))
>       if (!(authType.equals(KerberosAuthenticationHandler.TYPE))) {
>         filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           authType);
>         LOG.info("AuthType: "+authType);
>       } else {
>         filterConfig.put(AuthenticationFilter.AUTH_TYPE,
>           KerberosDelegationTokenAuthenticationHandler.class.getName());
>         LOG.info("AuthType: KerberosDelegationTokenAuthenticationHandler");
>       } 
>       // Resolve _HOST into bind address
>       String bindAddress = conf.get(HttpServer2.BIND_ADDRESS);
>       String principal =
>           filterConfig.get(KerberosAuthenticationHandler.PRINCIPAL);
>       if (principal != null) {
>         try {
>           principal = SecurityUtil.getServerPrincipal(principal, bindAddress);
>         } catch (IOException ex) {
>           throw new RuntimeException(
>               "Could not resolve Kerberos principal name: " + ex.toString(), ex);
>         }
>         filterConfig.put(KerberosAuthenticationHandler.PRINCIPAL,
>             principal);
>       }
>     }
>  {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message