nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [nifi] jfrazee commented on pull request #4216: NIFI-7356 Enable TLS for embedded Zookeeper when NiFi has TLS enabled
Date Fri, 01 May 2020 19:52:36 GMT

jfrazee commented on pull request #4216:
URL: https://github.com/apache/nifi/pull/4216#issuecomment-622538128


   @natural I think we're getting closer on this. I did a deep dive on it and there are some
failing configuration scenarios I think we need to handle. Also, I'm not seeing the secure
port actually being secured.
   
   **ZooKeeper client and quorum TLS enabled by hand in zookeeper.properties**
   
   This fails. I don't have a strong opinion about what the behavior should be beyond working.
I think the source of this one is when we remove the `clientPort` but it's set in the connect
string and we don't override it.
   
   ```
   2020-04-30 20:28:12,678 WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Invalid configuration
detected: secure NiFi with embedded ZooKeeper configured for unsecured HTTP connections.
   2020-04-30 20:28:12,678 WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Removed HTTP
port from embedded ZooKeeper configuration to deactivate insecure HTTP connections.
   2020-04-30 20:28:12,680 WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Potential mismatch
between NiFi ZK client connection string and embedded ZK server secure port.
   2020-04-30 20:28:12,680 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer NiFi properties
not mapped to ZooKeeper properties, all properties already set.
   2020-04-30 20:28:12,681 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Starting Embedded
ZooKeeper Peer
   2020-04-30 20:28:12,684 INFO [main] o.a.z.server.persistence.FileTxnSnapLog zookeeper.snapshot.trust.empty
: false
   2020-04-30 20:28:12,785 INFO [main] o.a.zookeeper.server.ServerCnxnFactory Using org.apache.zookeeper.server.NettyServerCnxnFactory
as server connection factory
   2020-04-30 20:28:12,795 INFO [main] o.a.z.server.persistence.FileTxnSnapLog zookeeper.snapshot.trust.empty
: false
   2020-04-30 20:28:12,805 INFO [main] org.apache.zookeeper.server.ZKDatabase zookeeper.snapshotSizeFactor
= 0.33
   2020-04-30 20:28:12,807 INFO [main] o.a.z.server.persistence.FileSnap Reading snapshot
/opt/nifi/state/zookeeper/version-2/snapshot.100000010
   2020-04-30 20:28:12,822 INFO [main] o.a.z.server.NettyServerCnxnFactory binding to port
null
   2020-04-30 20:28:12,824 WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup
of context o.e.j.w.WebAppContext@3382cf68{nifi-api,/nifi-api,file:///opt/nifi-1.12.0-SNAPSHOT/work/jetty/nifi-web-api-1.12.0-SNAPSHOT.war/webapp/,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.12.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.12.0-SNAPSHOT.war}
   org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
   ```
   
   **No ZooKeeper client or quorum TLS in zookeeper.properties**
   
   This works, but AFAICT isn't secure because we don't configure the connection factory.
I tested for that using testssl.sh.
   
   ```
   2020-04-30 20:32:01,722 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Starting Embedded
ZooKeeper Peer
   2020-04-30 20:32:01,724 INFO [main] o.a.z.server.persistence.FileTxnSnapLog zookeeper.snapshot.trust.empty
: false
   2020-04-30 20:32:01,816 INFO [main] o.a.zookeeper.server.ServerCnxnFactory Using org.apache.zookeeper.server.NettyServerCnxnFactory
as server connection factory
   2020-04-30 20:32:01,827 INFO [main] o.a.z.server.persistence.FileTxnSnapLog zookeeper.snapshot.trust.empty
: false
   2020-04-30 20:32:01,839 INFO [main] org.apache.zookeeper.server.ZKDatabase zookeeper.snapshotSizeFactor
= 0.33
   2020-04-30 20:32:01,842 INFO [main] o.a.z.server.persistence.FileSnap Reading snapshot
/opt/nifi/state/zookeeper/version-2/snapshot.100000010
   2020-04-30 20:32:01,863 INFO [main] o.a.z.server.NettyServerCnxnFactory binding to port
/0.0.0.0:2181
   2020-04-30 20:32:01,936 INFO [main] o.a.z.server.NettyServerCnxnFactory bound to port 2181
   2020-04-30 20:32:01,952 INFO [main] o.apache.nifi.controller.FlowController Checking if
there is already a Cluster Coordinator Elected...
   2020-04-30 20:32:02,022 INFO [main] org.apache.curator.utils.Compatibility Using emulated
InjectSessionExpiration
   2020-04-30 20:32:02,072 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
   2020-04-30 20:32:02,083 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer
value is 4194304 Bytes
   2020-04-30 20:32:02,108 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema
   ...
   2020-04-30 20:35:30,927 WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Invalid configuration
detected: secure NiFi with embedded ZooKeeper configured for unsecured HTTP connections.
   2020-04-30 20:35:30,927 WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Removed HTTP
port from embedded ZooKeeper configuration to deactivate insecure HTTP connections.
   2020-04-30 20:35:30,929 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Secure client
port set from NiFi ZK connection string, set port 2181
   2020-04-30 20:35:30,930 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.keystore' to ZooKeeper property 'ssl.keyStore.location' with value '/opt/nifi/conf/keystore.jks'
   2020-04-30 20:35:30,930 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.keystorePasswd' to ZooKeeper property 'ssl.keyStore.password' with value '********'
   2020-04-30 20:35:30,930 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.truststorePasswd' to ZooKeeper property 'ssl.trustStore.password' with value
'********'
   2020-04-30 20:35:30,930 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.truststore' to ZooKeeper property 'ssl.trustStore.location' with value '/opt/nifi/conf/truststore.jks'
   ```
   
   **secureClientPort set in zookeeper.properties without any other ZooKeeper client or quorum
TLS configuration***
   
   This seems to fail for clusters because of the way we're getting the network address: https://github.com/apache/nifi/pull/4216/files#diff-498c00ebd0253e4653debfcb926a913aR305-R307
   
   ```
   2020-05-01 15:05:49,675 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Embedded ZooKeeper
not configured for unsecured HTTP connections.
   2020-05-01 15:05:49,678 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Matched ZK client
connection string 77012757a73d:2281,41bb25eac337:2281,7cc0ba8a3b6e:2281 with embedded ZK server
secure port: 0.0.0.0/0.0.0.0:2281
   2020-05-01 15:05:49,681 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.keystore' to ZooKeeper property 'ssl.keyStore.location' with value '/opt/nifi/conf/keystore.jks'
   2020-05-01 15:05:49,681 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.keystorePasswd' to ZooKeeper property 'ssl.keyStore.password' with value '********'
   2020-05-01 15:05:49,681 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.truststorePasswd' to ZooKeeper property 'ssl.trustStore.password' with value
'********'
   2020-05-01 15:05:49,681 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Mapped NiFi property
'nifi.security.truststore' to ZooKeeper property 'ssl.trustStore.location' with value '/opt/nifi/conf/truststore.jks'
   2020-05-01 15:05:49,682 INFO [main] o.a.n.c.s.server.ZooKeeperStateServer Starting Embedded
ZooKeeper Peer
   …
   2020-05-01 15:05:50,026 INFO [main] o.a.z.server.NettyServerCnxnFactory binding to port
null            
   …                                            
   2020-05-01 15:05:50,030 WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup
of context o.e.j.w.WebAppContext@6cd98a05{nifi-api,/nifi-a
   org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.               
                                                             
   …
   Caused by: java.lang.NullPointerException: localAddress
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message