nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: Problem of nifi-spark-receiver with secured nifi
Date Fri, 09 Sep 2016 01:53:10 GMT
Hi Hang,

I do not have extensive experience with the Spark receiver, but I believe it runs on top of
the site-to-site raw socket protocol. When NiFi is secured and site-to-site is marked as secure,
site-to-site communications require mutual authentication TLS — each endpoint must have
a valid certificate which it presents to enable the other endpoint to authenticate it.

It appears that the certificate your “client” (in this case the spark receiver) is presenting
when it attempts to make a connection isn’t accepted by NiFi because the signature is not
valid. Does this certificate identify the correct hostname (or SAN entry) and has it been
modified at all? You can print additional debug information in the NiFi logs by starting the
application with an additional line in bootstrap.conf:

java.arg.15=-Djavax.net.debug=ssl,handshake

(You can replace ’15’ with a different number if this conflicts with an existing argument).

You can also verify that the certificate being used would be accepted by running this command:

$ openssl s_client -connect <host:port> -debug -state -cert <path_to_your_cert.pem>
-key <path_to_your_key.pem> -CAfile <path_to_your_CA_cert.pem>

The host and port are for the NiFi resource the NiFi Spark receiver is trying to connect to,
the cert is the certificate in the keystore, the key is the private key for that certificate,
and CAfile is the public key certificate of the remote resource or the CA that signed it.

If you were not providing a client certificate for the Spark receiver, you can do this using
SiteToSiteClient.Builder#sslContext if you have an existing SSLContext, or #keystoreFilename,
#keystorePass, etc. to provide those parameters. If the application is residing on the same
machine and you don’t have a separate certificate for this logical entity, it would be acceptable
to re-use the certificate in the keystore that NiFi is using (as long as that certificate
is trusted in the NiFi truststore).

My guess is that the certificate was generated with a different DN than what is presented
on the outgoing connection. I hope this helps. Please let us know if you cannot resolve the
issue.


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Sep 8, 2016, at 4:26 PM, Peng, Hang (CWM-NR) <hang.peng@rbc.com> wrote:
> 
> Hello Sir/Madam,
> 
> I use nifi 0.7 and I run it in on a single node. Previously, without any security setup,
my spark application was able to consume from Nifi by the means of nifi-spark-receiver (version
0.7.0). However, after I've secured nifi, my spark application cannot receive any data from
Nifi. We have already changed the siteToSiteClient.buildConfig in our code by adding following
properties:
> *       keystoreFilename
> *       keystorePass
> *       keyStoreTypeKey
> *       truststoreFilename
> *       truststorePass
> *       truststoreType
> 
> We receive following exception
> 
> WARN ReceiverSupervisorImpl: Restarting receiver with delay 2000 ms: Failed to receive
data from NiFi:
> javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX
path validation failed: java.security.cert.CertPathValidatorException: signature check failed
>        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
>        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
>        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
>        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
>        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
>        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
>        at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
>        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
>        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
>        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
>        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
>        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
>        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
>        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
>        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
>        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
>        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
>        at org.apache.nifi.remote.util.NiFiRestApiUtil.getController(NiFiRestApiUtil.java:66)
>        at org.apache.nifi.remote.client.socket.EndpointConnectionPool.refreshRemoteInfo(EndpointConnectionPool.java:887)
>        at org.apache.nifi.remote.client.socket.EndpointConnectionPool.getPortIdentifier(EndpointConnectionPool.java:874)
>        at org.apache.nifi.remote.client.socket.EndpointConnectionPool.getOutputPortIdentifier(EndpointConnectionPool.java:858)
>        at org.apache.nifi.remote.client.socket.SocketClient.getPortIdentifier(SocketClient.java:81)
>        at org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:123)
>        at org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable.run(NiFiReceiver.java:149)
>        at java.lang.Thread.run(Thread.java:745)
> Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException:
signature check failed
>        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
>        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:260)
>        at sun.security.validator.Validator.validate(Validator.java:260)
>        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
>        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
>        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
>        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
>        ... 21 more
> Caused by: java.security.cert.CertPathValidatorException: signature check failed
>        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
>        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
>        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
>        at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
>        at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
>        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
>        ... 27 more
> Caused by: java.security.SignatureException: Signature does not match.
>        at sun.security.x509.X509CertImpl.verify(X509CertImpl.java:424)
>        at sun.security.provider.certpath.BasicChecker.verifySignature(BasicChecker.java:166)
>        at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:147)
>        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
>        ... 32 more
> 
> Any help or suggest is much appreciated.
> 
> Regards,
> Hang Peng
> 
> 
> ____________________________________
> This e-mail may be privileged and/or confidential, and the sender does not waive any
related rights and obligations. Any distribution, use or copying of this e-mail or the information
it contains by other than an intended recipient(s) is unauthorized. If you received this e-mail
in error, please advise me (by return e-mail or otherwise) immediately and delete this e-mail.
> 
> 	The contents of any attachment to this e-mail may contain software viruses or other
defect which might affect your own computer system once received or opened. While companies
under RBC Investor Services Trust and its affiliates take reasonable precautions to minimize
that risk, we cannot accept liability or responsibility for any damage or loss which may occur
or be sustained as a result of a software virus or other defect. You are responsible for virus
checks before opening any attachment.


Mime
View raw message