hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kitti Nanasi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15609) Retry KMS calls when SSLHandshakeException occurs
Date Mon, 16 Jul 2018 11:20:00 GMT
Kitti Nanasi created HADOOP-15609:
-------------------------------------

             Summary: Retry KMS calls when SSLHandshakeException occurs
                 Key: HADOOP-15609
                 URL: https://issues.apache.org/jira/browse/HADOOP-15609
             Project: Hadoop Common
          Issue Type: Improvement
          Components: common, kms
            Reporter: Kitti Nanasi
            Assignee: Kitti Nanasi


KMS call should retry when javax.net.ssl.SSLHandshakeException occurs and FailoverOnNetworkExceptionRetry
policy is used.

For example in the following stack trace, we can see that the KMS Provider's connection is
lost, an SSLHandshakeException is thrown and the operation is not retried:
{code}
W0711 18:19:50.213472  1508 LoadBalancingKMSClientProvider.java:132] KMS provider at [https://example.com:16000/kms/v1/]
threw an IOException:
Java exception follows:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:512)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:502)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:791)
        at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:288)
        at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:284)
        at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:124)
        at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:284)
        at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:532)
        at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:927)
        at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:946)
        at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:316)
        at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:311)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:323)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
        at sun.security.ssl.InputRecord.read(InputRecord.java:505)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
        ... 22 more
W0711 18:19:50.239328  1508 LoadBalancingKMSClientProvider.java:149] Aborting since the Request
has failed with all KMS providers(depending on hadoop.security.kms.client.failover.max.retries=1
setting and numProviders=1) in the group OR the exception is not recoverable
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message