nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan H <ryan.howell.developm...@gmail.com>
Subject Re: NiFi 1.1.1 Secure Cluster Setup Issue: HTTPS Wrong Hostname wrong should be: <my.ip.address>
Date Wed, 01 Mar 2017 17:35:12 GMT
Hi Bryan,

Thank you for the quick response. I went ahead and tried using the hostname
instead of the actail IP address and that got me past the issue I was stuck
on. I am going to work a little bit more with it now that I have moved past
the blocking problem. I will send an update with my findings.

As of now, the one thing that I am seeing is that I can access the UI of
the secure cluster, however that is about it. My next step was to configure
the users and policies from my Initial Login Identity, but the request
doesn't seem to take and the UI for the given menu option just stays white
and never populates (just spins as if it is loading, but never loads).

I will follow up with what I did in detail once I have moved past the issue
so other users that may run in to the problem have a documented solution.

Cheers,

Ryan

On Wed, Mar 1, 2017 at 9:26 AM, Bryan Bende <bbende@gmail.com> wrote:

> Ryan,
>
> The general problem looks like your certs are not being created
> correctly for the specific hosts in your cluster. Each keystore.jks
> needs to be specific to the host that it is being used on.
>
> There are a couple of things in play here...
>
> - The Certificate Authority (CA) which is used to sign all the
> certificates so they can be trusted
> - The certificate for each NiFi host which needs to be generated for
> each hostname and signed by the CA
> - The truststore for each NiFi instance which will have the CA cert in
> it so that NiFi can trust incoming certs issued by the same CA
> - The client certificate which also needs to be signed by the CA,
> which you can use in p12 format in your browser to access the
> application
>
> Lets say you want a three node secure cluster and your hosts are
> host1, host2, and host3, you would need the following:
>
> keystore.jks on host1 with CN=host1, OU=NiFi
> keystore.jks on host2 with CN=host2, OU=NiFi
> keystore.jks on host3 with CN=host3, OU=NiFi
>
> This way when you go to https://host1:8443/nifi the hostname your
> requesting is the same as the host name in the keystore.jks being used
> by that instance.
>
> I'm not totally sure if using IP addresses will work here, maybe
> someone else can chime in on that, but typically I have seen people
> using hostnames.
>
> There are many ways to set this up...Pierre has a blog post that shows
> how to do it using the toolkit in client-server mode:
> https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-
> secured-cluster-setup/
>
> Hope that helps.
>
> -Bryan
>
>
> On Wed, Mar 1, 2017 at 8:43 AM, Ryan H
> <ryan.howell.development@gmail.com> wrote:
> > Dear NiFi Dev Team,
> >
> > I am currently in the process of setting up a secure NiFi Cluster and
> > running into an issue that I cannot resolve (almost a couple weeks
> now)--any
> > help will be greatly appreciated.
> >
> > As background, I have followed these articles (among others) to set up
> the
> > secure cluster:
> > https://community.hortonworks.com/articles/886/securing-
> nifi-step-by-step.html
> > http://bryanbende.com/development/2016/08/17/apache-
> nifi-1-0-0-authorization-and-multi-tenancy
> >
> > A thread that I read thru and followed to try and correct the issue
> (which
> > looks to be the same error I am getting, but does not resolve the
> problem):
> > http://apache-nifi.1125220.n5.nabble.com/Clustering-
> configuration-error-HTTPS-hostname-wrong-td13680.html
> >
> > What I have been able to do:
> > I have been able to set up a secure cluster of 3 nodes on my local
> machine
> > (using the NiFI Toolkit for cert/truststore/keystore generation).
> > I have been able to set up a standalone (not clustered) secure NiFi
> instance
> > in my target environment (AWS farm).
> > I have been able to set up an unsecured cluster in my target environment
> > (the same AWS farm).
> >
> > What I have not been able to do (which is what I am trying to
> accomplish):
> > I have not been able to setup a secure cluster in my target environment.
> I
> > have started with just a single machine in my AWS farm with a single node
> > (same machine I was able to set up a secure standalone instance of
> NiFI). I
> > have been trying to get just a single node as a secure cluster which I
> would
> > then replicate across the other machines in the farm once successful.
> >
> > I have tried various methods: NiFi Toolkit, TinyCert, and self signed
> > certificates which I have generated by hand via openssl.
> >
> > The current certs/truststore/keystore have all been generated using the
> NiFi
> > Toolkit version 1.1.1 and the version of NiFi I am running is 1.1.1
> >
> > Below are my relevant config settings and the resultant log output which
> is
> > the point at which I am stuck. Any help is very much appreciated. As an
> FYI,
> > everywhere below that says "my.ip.address" is the actual IP address of my
> > AWS server.
> >
> > Thanks in Advance,
> >
> > Ryan H.
> >
> >
> >
> >
> > "nifi.properties"
> >
> > # Site to Site properties
> > nifi.remote.input.host=my.ip.address
> > nifi.remote.input.secure=true
> > nifi.remote.input.socket.port=10443
> > nifi.remote.input.http.enabled=true
> > nifi.remote.input.http.transaction.ttl=30 sec
> >
> > # web properties #
> > nifi.web.war.directory=./lib
> > nifi.web.http.host=
> > nifi.web.http.port=
> > nifi.web.https.host=my.ip.address
> > nifi.web.https.port=443
> > nifi.web.jetty.working.directory=./work/jetty
> > nifi.web.jetty.threads=200
> >
> > # security properties #
> > nifi.sensitive.props.key=
> > nifi.sensitive.props.key.protected=
> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> > nifi.sensitive.props.provider=BC
> > nifi.sensitive.props.additional.keys=
> >
> > nifi.security.keystore=./conf/keystore.jks
> > nifi.security.keystoreType=jks
> > nifi.security.keystorePasswd=SiPE4sd4he5pI0kvmifB5DNnGTp/nnfGFVLO/TsVsIs
> > nifi.security.keyPasswd=SiPE4sd4he5pI0kvmifB5DNnGTp/nnfGFVLO/TsVsIs
> > nifi.security.truststore=./conf/truststore.jks
> > nifi.security.truststoreType=jks
> > nifi.security.truststorePasswd=Hs4tnGAAnywGF2Xo2+
> EKaJGlALMZcpJkzbZ/gdYcB2s
> > nifi.security.needClientAuth=
> > nifi.security.user.authorizer=file-provider
> > nifi.security.user.login.identity.provider=
> > nifi.security.ocsp.responder.url=
> > nifi.security.ocsp.responder.certificate=
> >
> > # cluster common properties (all nodes must have same values) #
> > nifi.cluster.protocol.heartbeat.interval=5 sec
> > nifi.cluster.protocol.is.secure=true
> >
> > # cluster node properties (only configure for cluster nodes) #
> > nifi.cluster.is.node=true
> > nifi.cluster.node.address=my.ip.address
> > nifi.cluster.node.protocol.port=11443
> > nifi.cluster.node.protocol.threads=10
> > nifi.cluster.node.event.history.size=25
> > nifi.cluster.node.connection.timeout=5 sec
> > nifi.cluster.node.read.timeout=5 sec
> > nifi.cluster.firewall.file=
> > nifi.cluster.flow.election.max.wait.time=2 mins
> > nifi.cluster.flow.election.max.candidates=
> >
> > # zookeeper properties, used for cluster management #
> > nifi.zookeeper.connect.string=my.ip.address:2181
> > nifi.zookeeper.connect.timeout=3 secs
> > nifi.zookeeper.session.timeout=3 secs
> > nifi.zookeeper.root.node=/nifi
> >
> > -------------------------------------------------
> >
> > "authorizers.xml"
> >
> > <authorizer>
> >         <identifier>file-provider</identifier>
> >         <class>org.apache.nifi.authorization.FileAuthorizer</class>
> >         <property name="Authorizations
> > File">./conf/authorizations.xml</property>
> >         <property name="Users File">./conf/users.xml</property>
> >         <property name="Initial Admin Identity">CN=admin,
> OU=NIFI</property>
> >         <property name="Legacy Authorized Users File"></property>
> >
> >         <property name="Node Identity 1">CN=my.ip.address,
> > OU=NIFI</property>
> >         <property name="Node Identity 2">CN=my.ip.address,
> > OU=NIFI</property>
> >         <property name="Node Identity 3">CN=my.ip.address,
> > OU=NIFI</property>
> >     </authorizer>
> >
> > -------------------------------------------------
> >
> > Output of ‘keytool -list -keystore
> > nifi-toolkit-1.1.1/target/CN\=admin_OU\=NIFI.p12 -storepass <password>
> > -storetype PKCS12 -v’
> >
> > Keystore type: PKCS12
> > Keystore provider: SunJSSE
> >
> > Your keystore contains 1 entry
> >
> > Alias name: nifi-key
> > Creation date: Feb 28, 2017
> > Entry type: PrivateKeyEntry
> > Certificate chain length: 2
> > Certificate[1]:
> > Owner: CN=admin, OU=NIFI
> > Issuer: CN=localhost, OU=NIFI
> > Serial number: 15a8674765e00000000
> > Valid from: Tue Feb 28 20:41:03 UTC 2017 until: Fri Feb 28 20:41:03 UTC
> 2020
> > Certificate fingerprints:
> >      MD5:  ED:2A:AD:BD:88:C4:B8:49:80:EA:58:A2:A4:83:FD:67
> >      SHA1: 19:2F:3C:41:43:B0:A7:3C:C8:4D:1F:A6:11:DC:FA:EC:61:C0:AC:6B
> >      SHA256:
> > 85:9F:E3:E6:2B:94:60:48:55:0B:5F:BF:55:DF:FC:C8:E8:30:CD:85:
> 3B:2B:F6:9A:B9:56:5C:EC:2F:43:C8:42
> >      Signature algorithm name: SHA256withRSA
> >      Version: 3
> >
> > Extensions:
> >
> > #1: ObjectId: 2.5.29.35 Criticality=false
> > AuthorityKeyIdentifier [
> > KeyIdentifier [
> > 0000: E1 FC AB 2F 69 4D 6A FE   A5 E8 2F B6 43 13 79 1C  .../iMj.../.C.y.
> > 0010: 46 F5 23 91                                        F.#.
> > ]
> > ]
> >
> > #2: ObjectId: 2.5.29.19 Criticality=false
> > BasicConstraints:[
> >   CA:false
> >   PathLen: undefined
> > ]
> >
> > #3: ObjectId: 2.5.29.37 Criticality=false
> > ExtendedKeyUsages [
> >   clientAuth
> >   serverAuth
> > ]
> >
> > #4: ObjectId: 2.5.29.15 Criticality=true
> > KeyUsage [
> >   DigitalSignature
> >   Non_repudiation
> >   Key_Encipherment
> >   Data_Encipherment
> >   Key_Agreement
> > ]
> >
> > #5: ObjectId: 2.5.29.14 Criticality=false
> > SubjectKeyIdentifier [
> > KeyIdentifier [
> > 0000: 69 E2 30 6B 51 4D 8F 5F   9D 96 B1 A2 17 64 AC 72  i.0kQM._.....d.r
> > 0010: A6 37 0E 9F                                        .7..
> > ]
> > ]
> >
> > Certificate[2]:
> > Owner: CN=localhost, OU=NIFI
> > Issuer: CN=localhost, OU=NIFI
> > Serial number: 15a8674717300000000
> > Valid from: Tue Feb 28 20:41:02 UTC 2017 until: Fri Feb 28 20:41:02 UTC
> 2020
> > Certificate fingerprints:
> >      MD5:  83:47:1D:17:85:3E:EC:5D:7B:B8:75:24:C3:F8:86:75
> >      SHA1: 2C:F1:71:7D:2F:67:B7:C6:C5:28:11:EC:A3:F2:51:D7:B3:07:F9:92
> >      SHA256:
> > 7C:63:31:EB:4A:E7:B1:26:C1:F5:67:89:46:90:EE:9C:5E:DD:B5:47:
> 45:4A:FB:F9:06:0F:81:9B:3B:CB:F4:E7
> >      Signature algorithm name: SHA256withRSA
> >      Version: 3
> >
> > Extensions:
> >
> > #1: ObjectId: 2.5.29.35 Criticality=false
> > AuthorityKeyIdentifier [
> > KeyIdentifier [
> > 0000: E1 FC AB 2F 69 4D 6A FE   A5 E8 2F B6 43 13 79 1C  .../iMj.../.C.y.
> > 0010: 46 F5 23 91                                        F.#.
> > ]
> > ]
> >
> > #2: ObjectId: 2.5.29.19 Criticality=false
> > BasicConstraints:[
> >   CA:true
> >   PathLen:2147483647
> > ]
> >
> > #3: ObjectId: 2.5.29.37 Criticality=false
> > ExtendedKeyUsages [
> >   clientAuth
> >   serverAuth
> > ]
> >
> > #4: ObjectId: 2.5.29.15 Criticality=true
> > KeyUsage [
> >   DigitalSignature
> >   Non_repudiation
> >   Key_Encipherment
> >   Data_Encipherment
> >   Key_Agreement
> >   Key_CertSign
> >   Crl_Sign
> > ]
> >
> > #5: ObjectId: 2.5.29.14 Criticality=false
> > SubjectKeyIdentifier [
> > KeyIdentifier [
> > 0000: E1 FC AB 2F 69 4D 6A FE   A5 E8 2F B6 43 13 79 1C  .../iMj.../.C.y.
> > 0010: 46 F5 23 91                                        F.#.
> > ]
> > ]
> >
> > *******************************************
> > *******************************************
> >
> > Output of "nifi-app.log"
> >
> > o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request
> GET
> > /nifi-api/flow/current-user to my.ip.address:443 due to {}
> > com.sun.jersey.api.client.ClientHandlerException: java.io.IOException:
> HTTPS
> > hostname wrong:  should be <my.ip.address>
> >     at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:155)
> > ~[jersey-client-1.19.jar:1.19]
> >     at com.sun.jersey.api.client.Client.handle(Client.java:652)
> > ~[jersey-client-1.19.jar:1.19]
> >     at
> > com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(
> GZIPContentEncodingFilter.java:123)
> > ~[jersey-client-1.19.jar:1.19]
> >     at com.sun.jersey.api.client.WebResource.handle(
> WebResource.java:682)
> > ~[jersey-client-1.19.jar:1.19]
> >     at com.sun.jersey.api.client.WebResource.access$200(
> WebResource.java:74)
> > ~[jersey-client-1.19.jar:1.19]
> >     at
> > com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
> > ~[jersey-client-1.19.jar:1.19]
> >     at
> > org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.
> java:578)
> > ~[nifi-framework-cluster-1.1.1.jar:1.1.1]
> >     at
> > org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator$NodeHttpRequest.run(
> ThreadPoolRequestReplicator.java:770)
> > ~[nifi-framework-cluster-1.1.1.jar:1.1.1]
> >     at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [na:1.8.0_121]
> >     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > [na:1.8.0_121]
> >     at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> > [na:1.8.0_121]
> >     at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> > [na:1.8.0_121]
> >     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> > Caused by: java.io.IOException: HTTPS hostname wrong:  should be
> > <my.ip.address>
> >     at
> > sun.net.www.protocol.https.HttpsClient.checkURLSpoofing(
> HttpsClient.java:649)
> > ~[na:1.8.0_121]
> >     at
> > sun.net.www.protocol.https.HttpsClient.afterConnect(
> HttpsClient.java:573)
> > ~[na:1.8.0_121]
> >     at
> > sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(
> AbstractDelegateHttpsURLConnection.java:185)
> > ~[na:1.8.0_121]
> >     at
> > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(
> HttpURLConnection.java:1546)
> > ~[na:1.8.0_121]
> >     at
> > sun.net.www.protocol.http.HttpURLConnection.getInputStream(
> HttpURLConnection.java:1474)
> > ~[na:1.8.0_121]
> >     at
> > java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> > ~[na:1.8.0_121]
> >     at
> > sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(
> HttpsURLConnectionImpl.java:338)
> > ~[na:1.8.0_121]
> >     at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(
> URLConnectionClientHandler.java:253)
> > ~[jersey-client-1.19.jar:1.19]
> >     at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:153)
> > ~[jersey-client-1.19.jar:1.19]
> >     ... 12 common frames omitted
> >
> > -------------------------------------------------
> >
> > Output of "nifi-user.log"
> >
> > 2017-02-28 21:03:47,763 INFO [NiFi Web Server-19]
> > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException:
> > Kerberos ticket login not supported by this NiFi.. Returning Conflict
> > response.
> > 2017-02-28 21:03:47,917 INFO [NiFi Web Server-16]
> > o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=admin,
> > OU=NIFI) GET https://1my.ip.address/nifi-api/flow/current-user (source
> ip:
> > my.machine.ip)
> > 2017-02-28 21:03:47,919 INFO [NiFi Web Server-16]
> > o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=admin,
> > OU=NIFI
> > 2017-02-28 21:05:44,793 INFO [NiFi Web Server-22]
> > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException:
> > Kerberos ticket login not supported by this NiFi.. Returning Conflict
> > response.
> > 2017-02-28 21:05:44,892 INFO [NiFi Web Server-20]
> > o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=admin,
> > OU=NIFI) GET https://my.ip.address/nifi-api/flow/current-user (source
> ip:
> > my.machine.ip)
> > 2017-02-28 21:05:44,893 INFO [NiFi Web Server-20]
> > o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=admin,
> > OU=NIFI
> >
> >
>

Mime
View raw message