Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
MIME-Version: 1.0
In-Reply-To: <CAEcou0QJ6n_5BYUZ_=5Fjvm407psph3-h91EiGFbJTx+sKo6Gg@mail.gmail.com>
References: <CAEcou0Q_PM+hTzC0fiziungrB2zzG=aBPsO+TjDBhffc=Fcifg@mail.gmail.com>
 <CAEcou0TWbznxZTT-qZFgnuVLwqkr8KSZiL6yB6dY5Y4rAqLA-w@mail.gmail.com>
 <3AE72AF8-88A1-4482-92A7-F5A21BDCAF0B@apache.org> <CAEcou0RX9RVgT-nviA25W6xX3_+O7nhKGD8L0sy81xoRg9D_JA@mail.gmail.com>
 <BLU437-SMTP19DB89E9035E8B78A93545CEA20@phx.gbl> <CAEcou0RvayMVGerGWZWLNFbPoRta3+YCbmamhJEBuqmtE_9VPA@mail.gmail.com>
 <40755F80-2A7D-497F-BC73-46DA43084A0D@apache.org> <CAEcou0QJ6n_5BYUZ_=5Fjvm407psph3-h91EiGFbJTx+sKo6Gg@mail.gmail.com>
From: Andre <andre-lists@fucs.org>
Date: Tue, 8 Nov 2016 20:31:47 +1100
Message-ID: <CAJpGy9LND6PPaMC+4q6WN-Ua5Z1t7xGeiroDpboGfdV9bww0CQ@mail.gmail.com>
Subject: Re: Secure Cluster Mode Issues
To: dev@nifi.apache.org
Content-Type: multipart/alternative; boundary=001a114b9d30438d9c0540c6cf77
archived-at: Tue, 08 Nov 2016 09:32:02 -0000

--001a114b9d30438d9c0540c6cf77
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Rick,

Can you confirm the certificate has a chain of trust with the default JDK
trusted certs? (i.e. trusted by the JVM)

Cheers


On Mon, Nov 7, 2016 at 3:38 AM, Ricky Saltzer <ricky@cloudera.com> wrote:

> Hey Andy -
>
> Thanks again for the help.
>
> The error message seems indicative that it doesn't seem to properly read
> the keystore file. One thing to note, if I point the nifi properties to a
> bogus keystore location, then it actually throws a FileNotFound exception=
.
> This is really odd behavior, because as I mentioned I'm able to start it =
in
> standalone mode using the correct keystore location, just as I try to do =
in
> clustered mode.
>
> I've attached both the clustered [1] nifi.properties, which doesn't work,
> and the standalone [2] which does work. . I restored it to a more basic
> configuration without the encrypted configuration, but with SSL still
> enabled. I also added a diff [3] of both the standalone and clustered
> properties file. Note that I I only have NiFi configured to use the
> keystore and not a truststore. I've redacted a few of the values in the
> property files, but be assured that the keystore is most definitely valid
> and is readable / locatable, as starting in standalone works just fine.
>
> I ran the SSL command [4] you gave me, minus the three PEM file arguments
> as I don't have any of those on hand. I hope that is fine. The output sti=
ll
> looks good.
>
> [1] https://gist.github.com/rickysaltzer/712aa6586592fe6628db2d57cec7a562
> [2] https://gist.github.com/rickysaltzer/fe11c8233e4434eacedd7fd0a083d950
> [3] https://gist.github.com/rickysaltzer/d715c7451eb554a54f14ec6e64da8558
> [4] https://gist.github.com/rickysaltzer/5d7cdeff8868bfc1f47010189735411a
>
>
>
>
> On Fri, Nov 4, 2016 at 7:48 PM, Andy LoPresto <alopresto@apache.org>
> wrote:
>
> > Hi Ricky,
> >
> > Sorry, should have noted that the debug output goes to
> nifi-bootstrap.log,
> > so thanks Mark for jumping in to help there.
> >
> > If you look at the top of that log, you=E2=80=99ll note that there is n=
o keystore
> > file provided and the truststore loaded is the default JRE cacerts
> > truststore. Can you please provide your nifi.properties file in a Gist,
> **taking
> > care to redact any sensitive values** like keystore/truststore password=
s,
> > although I think from looking at your log output, you are taking
> advantage
> > of the encrypted configuration feature, so even viewing the encrypted
> > values should be ok. Could you also please provide the directory listin=
g
> > where the keystore and truststore are located including the permissions
> and
> > ownership information?
> >
> > There may be a bug in the logic between cluster and standalone mode, bu=
t
> I
> > haven=E2=80=99t encountered this behavior before. If you can start NiFi=
 in
> > standalone mode, could you please provide the output of the following
> > command run from the terminal? It will simulate an HTTPS connection to
> the
> > server and verify the key and certificate presented by NiFi.
> >
> > * host =E2=80=94 the NiFi hostname
> > * port =E2=80=94 the port NiFi is running on
> > * path_to_your_cert.pem =E2=80=94 the public key certificate identifyin=
g the
> > client/user (i.e. what you load into your browser to authenticate)
> > * path_to_your_key.key =E2=80=94 the private key identifying the client=
/user
> > * path_to_your_CA_cert.pem =E2=80=94 the public key certificate identif=
ying the
> CA
> > which signed your NiFi server certificate (if self-signed, provide that
> > certificate)
> >
> > $ openssl s_client -connect <host:port> -debug -state -cert
> > <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile
> > <path_to_your_CA_cert.pem>
> >
> > Andy LoPresto
> > alopresto@apache.org
> > *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Nov 4, 2016, at 11:21 AM, Ricky Saltzer <ricky@cloudera.com> wrote:
> >
> > Hey guys -
> >
> > I went ahead and uploaded the boostrap log. I took a look at it and it
> > seems to be the same error [1]
> >
> > [1]:
> > https://gist.githubusercontent.com/rickysaltzer/
> > b156594f92066873f80928eddf84e7bb/raw/4d0e018038b332f4fdf14644910dfa
> > 9e70c57e49/gistfile1.txt
> >
> > On Fri, Nov 4, 2016 at 2:14 PM, Mark Payne <markap14@hotmail.com> wrote=
:
> >
> > Hey Ricky,
> >
> > When you enable debug logging for SSL, it writes to StdErr (or StdOut?)
> so
> > it will end up in your logs/nifi-bootstrap.log instead of nifi-app.log.
> > Can you give that a look?
> >
> > Thanks
> > -Mark
> >
> > On Nov 4, 2016, at 2:07 PM, Ricky Saltzer <ricky@cloudera.com> wrote:
> >
> > Hey Andy -
> >
> > Thanks for the response. I'm currently just trying to get one node in
> > clustered mode before adding a second. The keystore is stored locally a=
nd
> > I've confirmed it's readable, as it was able to start once I took it ou=
t
> >
> > of
> >
> > clustered mode. I added that line to the bootstrap.conf, but I don't
> > believe any additional logging was produced in regards to troubleshooti=
ng
> > this problem. Just in case, I've attached the entire log [1].
> >
> > [1]:
> > https://gist.githubusercontent.com/rickysaltzer/
> >
> > ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0
> > fedabc88bb/gistfile1.txt <https://gist.githubusercontent.com/
> rickysaltzer/
> > ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0
> > fedabc88bb/gistfile1.txt>
> >
> >
> > On Wed, Nov 2, 2016 at 7:08 PM, Andy LoPresto <alopresto@apache.org
> >
> > <mailto:alopresto@apache.org>> wrote:
> >
> >
> > Hi Ricky,
> >
> > Sorry to hear you are having this issue. Is the keystore available on
> >
> > all
> >
> > nodes of the cluster? It appears from the log message that the keystore
> >
> > is
> >
> > not found during startup. To further debug, you can add the following
> >
> > line
> >
> > in bootstrap.conf to provide additional logging:
> >
> > java.arg.15=3D-Djavax.net.debug=3Dssl,handshake
> >
> > Andy LoPresto
> > alopresto@apache.org <mailto:alopresto@apache.org>
> > *alopresto.apache@gmail.com <mailto:alopresto.apache@gmail.com> <
> >
> > alopresto.apache@gmail.com <mailto:alopresto.apache@gmail.com>>*
> >
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Nov 2, 2016, at 2:25 PM, Ricky Saltzer <ricky@cloudera.com> wrote:
> >
> > Hey all -
> >
> > I'm using NiFi 1.0 and I'm having an issue using secure mode with a
> >
> > local
> >
> > key store while in clustered mode. If I set the node in clustered mode,
> >
> > and
> >
> > also provide a valid keystore, I receive a KeyStoreException [1]. If I
> >
> > set
> >
> > the configuration to not use clustered mode, NiFi will start up fine
> >
> > with
> >
> > the provided key store. Am I supposed to be storing this key store in
> > Zookeeper somewhere?
> >
> >
> > [1]
> >
> >
> > Caused by: java.security.KeyStoreException:  not found
> >
> >
> >      at java.security.KeyStore.getInstance(KeyStore.java:839)
> > ~[na:1.8.0_11]
> >
> >      at
> > org.apache.nifi.io.socket.SSLContextFactory.<init>(
> > SSLContextFactory.java:61)
> > ~[nifi-socket-utils-1.0.0.jar:1.0.0]
> >
> >      at
> > org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto
> > ryBean.getObject(ServerSocketConfigurationFactoryBean.java:45)
> > ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0]
> >
> >      at
> > org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto
> > ryBean.getObject(ServerSocketConfigurationFactoryBean.java:30)
> > ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0]
> >
> >      at
> > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.
> > doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
> > ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> >
> >      ... 69 common frames omitted
> >
> > Caused by: java.security.NoSuchAlgorithmException:  KeyStore not
> >
> > available
> >
> >
> >      at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
> > ~[na:1.8.0_11]
> >
> >      at java.security.Security.getImpl(Security.java:695)
> >
> > ~[na:1.8.0_11]
> >
> >
> >      at java.security.KeyStore.getInstance(KeyStore.java:836)
> > ~[na:1.8.0_11]
> >
> >      ... 73 common frames omitted
> >
> >
> >
> >
> >
> > --
> > Ricky Saltzer
> > http://www.cloudera.com <http://www.cloudera.com/>
> >
> >
> >
> >
> >
> > --
> > Ricky Saltzer
> > http://www.cloudera.com
> >
> >
> >
>
>
> --
> Ricky Saltzer
> http://www.cloudera.com
>

--001a114b9d30438d9c0540c6cf77--