hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8485) Transparent Encryption Fails to work with Yarn/MapReduce
Date Wed, 27 May 2015 16:32:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561223#comment-14561223
] 

Xiaoyu Yao commented on HDFS-8485:
----------------------------------

Hi, [~ashar103], Thanks for reporting the issue. 
Can you add the Hadoop version you are using? This might be one of the known issues in 2.6
that has been fixed in 2.7.
Can you do a klist -kt /data/security/kms/keytab/kms.keytab to see if the configured kerberos
principle HTTP/kdchostname is there?

> Transparent Encryption Fails to work with Yarn/MapReduce
> --------------------------------------------------------
>
>                 Key: HDFS-8485
>                 URL: https://issues.apache.org/jira/browse/HDFS-8485
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: RHEL-7, Kerberos 5
>            Reporter: Ambud Sharma
>            Priority: Critical
>         Attachments: core-site.xml, hdfs-site.xml, kms-site.xml, mapred-site.xml, yarn-site.xml
>
>
> Running a simple MapReduce job that writes to a path configured as an encryption zone
throws exception
> 11:26:26,343 INFO  [org.apache.hadoop.mapreduce.Job] (pool-14-thread-1) Task Id : attempt_1432740034176_0001_m_000000_2,
Status : FAILED
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) Error: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:424)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:710)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1358)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1457)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1442)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:400)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:393)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> 11:26:26,346 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:393)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:337)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at com.s3.ingestion.S3ImportMR$S3ImportMapper.map(S3ImportMR.java:112)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at com.s3.ingestion.S3ImportMR$S3ImportMapper.map(S3ImportMR.java:43)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at java.security.AccessController.doPrivileged(Native
Method)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 11:26:26,347 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:127)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:418)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:413)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at java.security.AccessController.doPrivileged(Native
Method)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> 11:26:26,348 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:413)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	... 23 more
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) Caused by: GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at java.security.AccessController.doPrivileged(Native
Method)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 11:26:26,349 ERROR [stderr] (pool-14-thread-1) 	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261)
> 11:26:26,350 ERROR [stderr] (pool-14-thread-1) 	... 33 more
> 11:26:26,350 ERROR [stderr] (pool-14-thread-1) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message