nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Ward <ryan.wa...@gmail.com>
Subject Re: 1.11.3 trust store error
Date Tue, 03 Mar 2020 12:40:25 GMT
Endre  - thanks that was it

On Tue, Mar 3, 2020 at 6:50 AM Endre Kovacs
<andrewsmith87@protonmail.com.invalid> wrote:

> Hi,
>
> One additional thing:
>
> we encountered something strange as well:
>
> on 1.11.2 clustered, kerberized: request replication worked well.
>
> on 1.11.3 clustered, kerberized: request replication did not work, unless
> you specify, and set
> nifi.security.keyPasswd
>
> to the very same password as the
>
> nifi.security.keystorePasswd
>
> For us this resolved the issue.
>
> Best regards,
> Endre
>
> Sent with [ProtonMail](https://protonmail.com) Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, March 3, 2020 12:40 PM, Ryan Ward <ryan.ward2@gmail.com>
> wrote:
>
> > Hi Joe - Did you resolve your issue? If so I am wondering what the fix
> was as I'm seeing the same error on my cluster.
> >
> > On Thu, Feb 27, 2020 at 3:13 AM Endre Kovacs <
> andrewsmith87@protonmail.com.invalid> wrote:
> >
> >> Hi Joe,
> >>
> >> 1.  Have you tried connecting/debugging with openssl? From one pod to
> the other:
> >>     (openssl s_client -debug -CAfile
> ca-bundle-signing-node-certificates.crt -cert my-client-cert.crt -connect
> nifi-3.nifi-headless.lizardspock.svc.cluster.local:6007)
> >>
> >> 2. certs can also be verified by:
> >>  openssl verify -verbose -CAfile ca-bundle.crt my-client-cert.crt
> >>
> >> 3.  Can you check if no intermediary CAs are missing from the nodes
> truststore?
> >>
> >> 4.  This exception is coming from inter-node communication (replication
> of request from one node to the other). This means that it is unrelated to
> external user's authentication by client certificate. The question is: is
> your inter node communication secured by the trusted root CA (that you are
> sure that the CA cert is present in the trust store) or is it secured by
> selfsigned CA (which's CA may be lacking from your truststore)?
> >>
> >> 5.  `nifi.security.needClientAuth` is not part of NiFi properties any
> more. If SSL is turned on, and no
> `nifi.security.user.login.identity.provider` is set, then client cert based
> auth is the default. But supplying this property have no detrimental effect
> anyhow.
> >>
> >> Best regards,
> >> Endre
> >>
> >>     Sent with ProtonMail Secure Email.
> >>
> >>     ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> >>     On Wednesday, February 26, 2020 6:22 PM, Joe Gresock
> jgresock@gmail.com wrote:
> >>
> >>> Were there any changes with how the trust store is used in 1.11.3? I
> had a
> >>> 1.11.0 deployment working with the following settings, but when I
> deployed
> >>> 1.11.3, the cluster can't seem to replicate requests to itself:
> >>> nifi.remote.input.host=<redacted>
> >>> nifi.remote.input.secure=true
> >>> nifi.remote.input.socket.port=32440
> >>> nifi.remote.input.http.enabled=true
> >>> nifi.cluster.protocol.is.secure=true
> >>> nifi.cluster.is.node=true
> >>>
> nifi.cluster.node.address=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> >>> nifi.cluster.node.protocol.port=6007
> >>> nifi.web.https.host=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> >>> nifi.web.https.port=8443
> >>> nifi.security.keystore=./conf/keystore.jks
> >>> nifi.security.keystoreType=jks
> >>> nifi.security.keystorePasswd=<password>
> >>> nifi.security.keyPasswd=
> >>> nifi.security.truststore=./conf/truststore.jks
> >>> nifi.security.truststoreType=jks
> >>> nifi.security.truststorePasswd=<password>
> >>> nifi.security.needClientAuth=true
> >>> A trusted client cert that worked against the old cluster is getting
> the
> >>> same trust error (PKIX path building failed). I've verified that the
> >>> client cert was issued by an issuer that is definitely in the
> >>> ./conf/truststore.jks as a trustedCertEntry.
> >>> 2020-02-26 17:11:09,573 WARN [Replicate Request Thread-7]
> >>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> >>> javax.net.ssl.SSLHandshakeException:
> >>> sun.security.validator.ValidatorException: PKIX path building failed:
> >>> sun.security.provider.certpath.SunCertPathBuilderException: unable to
> find
> >>> valid certification path to r
> >>> equested target
> >>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> >>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
> >>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
> >>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
> >>> at
> >>>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
> >>> at
> >>>
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
> >>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
> >>> at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
> >>> at
> >>> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
> >>> at
> >>>
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
> >>> at
> >>> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
> >>> at
> >>> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
> >>> at
> >>>
> okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:302)
> >>> at
> >>>
> okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270)
> >>> at
> >>>
> okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162)
> >>> at
> >>>
> okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)
> >>> at
> >>>
> okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
> >>> at
> >>>
> okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
> >>> at
> >>>
> okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> >>> at
> >>>
> okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> >>> at
> >>>
> okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> >>> at
> >>>
> okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> >>> at
> >>>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> >>> at
> >>> okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
> >>> at okhttp3.RealCall.execute(RealCall.java:77)
> >>> at
> >>>
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:143)
> >>> at
> >>>
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:137)
> >>> at
> >>>
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647)
> >>> at
> >>>
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839)
> >>> at
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: sun.security.validator.ValidatorException: PKIX path
> building
> >>> failed: sun.security.provider.certpath.SunCertPathBuilderException:
> unable
> >>> to find valid certification path to requested target
> >>> at
> >>> sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
> >>> at
> >>>
> sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
> >>> at sun.security.validator.Validator.validate(Validator.java:262)
> >>> at
> >>>
> sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330)
> >>> at
> >>>
> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237)
> >>> at
> >>>
> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
> >>> at
> >>>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
> >>> ... 35 common frames omitted
> >>> Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
> >>> unable to find valid certification path to requested target
> >>> at
> >>>
> sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
> >>> at
> >>>
> sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
> >>> at
> >>> java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
> >>> at
> >>> sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
> >>> ... 41 common frames omitted
> >>> Thanks,
> >>> Joe

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message