hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13180) Encryption Zone data Run mr with execption:AuthenticationException can't be found in cache
Date Sat, 21 May 2016 05:03:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294716#comment-15294716
] 

Xiao Chen commented on HADOOP-13180:
------------------------------------

bq. Basically, your hadoop.kms.key.provider.uri in kms-site.xml should be something similar
to: kms://https@kms01.example.com;kms02.example.com:16000/kms. That is, use semicolon to separate
multiple kms instances.
One correction is the parameter is called {{dfs.encryption.key.provider.uri}}, and should
be set across-cluster (e.g. in hdfs-site.xml), not just kms-site.xml since the clients need
it as well.

bq. The cause of this problem is Multiple Instances of KMS, Behind a Load-Balancer or VIP
。 if only one Instances of KMS its OK, Multiple Instances of KMS not work, and throw the
above exeption logs.
Feels to me that this either means the load balancer behind the KMS needs to be further looked
at, or somehow multiple KMS' weren't setup correctly to share the secrets. The secret sharing
should be done using zookeeper. See https://hadoop.apache.org/docs/r2.6.2/hadoop-kms/index.html#HTTP_Authentication_Signature
for an example.

> Encryption Zone data Run mr  with execption:AuthenticationException  can't be found in
cache
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-13180
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13180
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.6.1
>            Reporter: lushuai
>
> org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.security.authentication.client.AuthenticationException:
org.apache.hadoop.security.token.SecretManager$InvalidToken: token (owner=hive, renewer=yarn,
realUser=, issueDate=1463627282514, maxDate=1464232082514, sequenceNumber=217, masterKeyId=2)
can't be found in cache
> 	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:249)
> 	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketForFileIdx(FileSinkOperator.java:622)
> 	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:566)
> 	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:675)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
> 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:163)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException:
org.apache.hadoop.security.token.SecretManager$InvalidToken: token (owner=hive, renewer=yarn,
realUser=, issueDate=1463627282514, maxDate=1464232082514, sequenceNumber=217, masterKeyId=2)
can't be found in cache
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> 	at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
> 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:487)
> 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:445)
> 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:719)
> 	at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
> 	at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1347)
> 	at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1446)
> 	at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1431)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:400)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:393)
> 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:393)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:337)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:801)
> 	at org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat.getHiveRecordWriter(HiveIgnoreKeyTextOutputFormat.java:80)
> 	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:261)
> 	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:246)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message