hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongzhi Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10098) HS2 local task for map join fails in KMS encrypted cluster
Date Thu, 26 Mar 2015 20:24:52 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382591#comment-14382591
] 

Yongzhi Chen commented on HIVE-10098:
-------------------------------------

The root cause, as pointed by Arun Suresh, when MapredLocalTask construct the command line
for the ChildJVM, it is calling the fs.getDelegationToken(uname) API. This unfortunately gets
ONLY the HDFS delegation token. It should be using fs.addDelegationTokens(uname, new Credentials())
this will also pick the KMS delegation token and store it in the file.

> HS2 local task for map join fails in KMS encrypted cluster
> ----------------------------------------------------------
>
>                 Key: HIVE-10098
>                 URL: https://issues.apache.org/jira/browse/HIVE-10098
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Yongzhi Chen
>            Assignee: Yongzhi Chen
>
> Env: KMS was enabled after cluster was kerberos secured. 
> Problem: PROBLEM: Any Hive query via beeline that performs a MapJoin fails with a java.lang.reflect.UndeclaredThrowableException
 from KMSClientProvider.addDelegationTokens.
> {code}
> 2015-03-18 08:49:17,948 INFO [main]: Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022))
- mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 
> 2015-03-18 08:49:19,048 WARN [main]: security.UserGroupInformation (UserGroupInformation.java:doAs(1645))
- PriviledgedActionException as:hive (auth:KERBEROS) cause:org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt) 
> 2015-03-18 08:49:19,050 ERROR [main]: mr.MapredLocalTask (MapredLocalTask.java:executeFromChildJVM(314))
- Hive Runtime Error: Map local work failed 
> java.io.IOException: java.io.IOException: java.lang.reflect.UndeclaredThrowableException

> at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:634) 
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.startForward(MapredLocalTask.java:363)

> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.startForward(MapredLocalTask.java:337)

> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:303)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:735) 
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

> at java.lang.reflect.Method.invoke(Method.java:606) 
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 
> Caused by: java.io.IOException: java.lang.reflect.UndeclaredThrowableException 
> at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:826)

> at org.apache.hadoop.crypto.key.KeyProviderDelegationTokenExtension.addDelegationTokens(KeyProviderDelegationTokenExtension.java:86)

> at org.apache.hadoop.hdfs.DistributedFileSystem.addDelegationTokens(DistributedFileSystem.java:2017)

> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:121)

> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)

> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)

> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:205) 
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:313) 
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:413)

> at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:559) 
> ... 9 more 
> Caused by: java.lang.reflect.UndeclaredThrowableException 
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1655)

> at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:808)

> ... 18 more 
> Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt) 
> at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306)

> at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196)

> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:127)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message