nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups
Date Wed, 19 Oct 2016 17:03:25 GMT
That is definitely weird that it is only an issue on the node that used to
be the NCM.
Might be worth double checking the keystore and truststore of that one node
and make sure it has what you would expect, and also double check
nifi.properties compared to the others to see if anything seems different.

Changing all of the keystores, truststores, etc should be fine from a data
perspective...

If you decide to go that route it would probably be easiest to start back
over from a security perspective, meaning:
- Stop all the nodes and delete the users.xml and authorizations.xml from
all nodes
- Configure authorizers.xml with the appropriate initial admin (or legacy
file) and node identities based on the new certs
- Ensure authorizers.xml is the same on all nodes
- Then restart everything

Alternatively, you might be able to manually add users for all of the new
certs before shutting everything down and give them the appropriate
policies, then restart everything, but requires more manual work to get
everything lined up.


On Wed, Oct 19, 2016 at 11:52 AM, Conrad Crampton <
conrad.crampton@secdata.com> wrote:

> Hi,
>
> As a plan for tomorrow – I have generated new keystores, truststores,
> client certts  etc. for all nodes in my cluster using the
>
>
>
> *From: *Bryan Bende <bbende@gmail.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Wednesday, 19 October 2016 at 15:33
>
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups
>
>
>
> Trying to think of things to check here...
>
>
>
> Does every node have nifi.remote.input.secure=true in nifi.properties and
> the URL in the RPG is an https URL?
>
>
>
> On Wed, Oct 19, 2016 at 10:25 AM, Conrad Crampton <
> conrad.crampton@secdata.com> wrote:
>
> One other thing…
>
> The RPGs have an unlocked padlock on them saying S2S is not secure.
>
> Conrad
>
>
>
> *From: *Bryan Bende <bbende@gmail.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Wednesday, 19 October 2016 at 15:20
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups
>
>
>
> Ok that does seem like a TLS/SSL issue...
>
>
>
> Is this a single cluster doing site-to-site to itself?
>
>
>
> On Wed, Oct 19, 2016 at 10:06 AM, Joe Witt <joe.witt@gmail.com> wrote:
>
> thanks conrad - did get it.  Bryan is being more helpful that I so I
> went silent :-)
>
> On Wed, Oct 19, 2016 at 10:02 AM, Conrad Crampton
>
> <conrad.crampton@secdata.com> wrote:
> > Hi Joe,
> >     Yep,
> >     Tried removing the RPG that referenced the NCM and adding new one
> with one of the datanodes as url.
> >     That sort of worked, but kept getting errors about the NCM not being
> available for the ports and therefore couldn’t actually enable the port I
> needed to for that RPG.
> >     Thanks
> >     Conrad
> >
> > (sending again as don’t know if the stupid header ‘spoofed’ is stopping
> getting though – apologies if already sent)
> >
> >     On 19/10/2016, 14:12, "Joe Witt" <joe.witt@gmail.com> wrote:
> >
> >         Conrad,
> >
> >         For s2s now you can just point at any of the nodes in the
> cluster.
> >         Have you tried changing the URL or removing and adding new RPG
> >         entries?
> >
> >         Thanks
> >         Joe
> >
> >         On Wed, Oct 19, 2016 at 8:38 AM, Conrad Crampton
> >         <conrad.crampton@secdata.com> wrote:
> >         > Hi,
> >         >
> >         > I have finally taken the plunge to upgrade my cluster from
> 0.6.1 to 1.0.0.
> >         >
> >         > 6 nodes with a NCM.
> >         >
> >         > With the removal of NCM in 1.0.0 I believe I now have an issue
> where none of
> >         > my Remote Process Groups work as they previously did because
> they were
> >         > configured to connect to the NCM (as the RPG url) which now
> doesn’t exist.
> >         >
> >         > I have tried converting my NCM to a node but whilst I can get
> it running
> >         > (sort of) when I try and connect to the cluster I get
> something like this in
> >         > my logs…
> >         >
> >         >
> >         >
> >         > 2016-10-19 13:14:44,109 ERROR [main] o.a.nifi.controller.
> StandardFlowService
> >         > Failed to load flow from cluster due to:
> >         > org.apache.nifi.controller.UninheritableFlowException: Failed
> to connect
> >         > node to cluster because local flow is different than cluster
> flow.
> >         >
> >         > org.apache.nifi.controller.UninheritableFlowException: Failed
> to connect
> >         > node to cluster because local flow is different than cluster
> flow.
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.
> loadFromConnectionResponse(StandardFlowService.java:879)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.load(
> StandardFlowService.java:493)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.web.server.JettyServer.start(JettyServer.
> java:746)
> >         > [nifi-jetty-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         > Caused by: org.apache.nifi.controller.UninheritableFlowException:
> Proposed
> >         > Authorizer is not inheritable by the flow controller because
> of Authorizer
> >         > differences: Proposed Authorizations do not match current
> Authorizations
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowSynchronizer.sync(
> StandardFlowSynchronizer.java:252)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.FlowController.synchronize(
> FlowController.java:1435)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.persistence.StandardXMLFlowConfigurationDA
> O.load(StandardXMLFlowConfigurationDAO.java:83)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.loadFromBytes(
> StandardFlowService.java:671)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.
> loadFromConnectionResponse(StandardFlowService.java:857)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         ... 4 common frames omitted
> >         >
> >         > 2016-10-19 13:14:44,414 ERROR [main] o.a.n.c.c.node.
> NodeClusterCoordinator
> >         > Event Reported for ncm-cm1.mis-cds.local:9090 -- Node
> disconnected from
> >         > cluster due to org.apache.nifi.controller.UninheritableFlowException:
> Failed
> >         > to connect node to cluster because local flow is different
> than cluster
> >         > flow.
> >         >
> >         > 2016-10-19 13:14:44,420 ERROR [Shutdown Cluster Coordinator]
> >         > org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> Thread[Shutdown
> >         > Cluster Coordinator,5,main]: java.lang.NullPointerException
> >         >
> >         > 2016-10-19 13:14:44,423 ERROR [Shutdown Cluster Coordinator]
> >         > org.apache.nifi.NiFi
> >         >
> >         > java.lang.NullPointerException: null
> >         >
> >         >         at
> >         > java.util.concurrent.ConcurrentHashMap.putVal(
> ConcurrentHashMap.java:1011)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at
> >         > java.util.concurrent.ConcurrentHashMap.put(
> ConcurrentHashMap.java:1006)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at
> >         > org.apache.nifi.cluster.coordination.node.
> NodeClusterCoordinator.updateNodeStatus(NodeClusterCoordinator.java:570)
> >         > ~[nifi-framework-cluster-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.cluster.coordination.node.
> NodeClusterCoordinator.shutdown(NodeClusterCoordinator.java:119)
> >         > ~[nifi-framework-cluster-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService$1.run(
> StandardFlowService.java:330)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at java.lang.Thread.run(Thread.java:745)
> ~[na:1.8.0_51]
> >         >
> >         > 2016-10-19 13:14:44,448 WARN [main] o.a.n.c.l.e.
> CuratorLeaderElectionManager
> >         > Failed to close Leader Selector for Cluster Coordinator
> >         >
> >         > java.lang.IllegalStateException: Already closed or has not
> been started
> >         >
> >         >         at
> >         > com.google.common.base.Preconditions.checkState(
> Preconditions.java:173)
> >         > ~[guava-18.0.jar:na]
> >         >
> >         >         at
> >         > org.apache.curator.framework.recipes.leader.LeaderSelector.
> close(LeaderSelector.java:270)
> >         > ~[curator-recipes-2.11.0.jar:na]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.leader.election.
> CuratorLeaderElectionManager.stop(CuratorLeaderElectionManager.java:159)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.FlowController.shutdown(
> FlowController.java:1303)
> >         > [nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.stop(
> StandardFlowService.java:339)
> >         > [nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.web.server.JettyServer.start(JettyServer.
> java:753)
> >         > [nifi-jetty-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         > 2016-10-19 13:14:45,062 WARN [Cluster Socket Listener]
> >         > org.apache.nifi.io.socket.SocketListener Failed to
> communicate with Unknown
> >         > Host due to java.net.SocketException: Socket closed
> >         >
> >         > java.net.SocketException: Socket closed
> >         >
> >         >         at java.net.PlainSocketImpl.socketAccept(Native
> Method)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at
> >         > java.net.AbstractPlainSocketImpl.accept(
> AbstractPlainSocketImpl.java:404)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at java.net.ServerSocket.implAccept(ServerSocket.java:
> 545)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at
> >         > sun.security.ssl.SSLServerSocketImpl.accept(
> SSLServerSocketImpl.java:348)
> >         > ~[na:1.8.0_51]
> >         >
> >         >         at
> >         > org.apache.nifi.io.socket.SocketListener$2.run(
> SocketListener.java:112)
> >         > ~[nifi-socket-utils-1.0.0.jar:1.0.0]
> >         >
> >         >         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> >         >
> >         > 2016-10-19 13:14:45,064 WARN [main] org.apache.nifi.web.server.
> JettyServer
> >         > Failed to start web server... shutting down.
> >         >
> >         > java.lang.Exception: Unable to load flow due to:
> java.io.IOException:
> >         > org.apache.nifi.controller.UninheritableFlowException: Failed
> to connect
> >         > node to cluster because local flow is different than cluster
> flow.
> >         >
> >         >         at
> >         > org.apache.nifi.web.server.JettyServer.start(JettyServer.
> java:755)
> >         > ~[nifi-jetty-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
> >         > [nifi-runtime-1.0.0.jar:1.0.0]
> >         >
> >         > Caused by: java.io.IOException:
> >         > org.apache.nifi.controller.UninheritableFlowException: Failed
> to connect
> >         > node to cluster because local flow is different than cluster
> flow.
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.load(
> StandardFlowService.java:497)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.web.server.JettyServer.start(JettyServer.
> java:746)
> >         > ~[nifi-jetty-1.0.0.jar:1.0.0]
> >         >
> >         >         ... 2 common frames omitted
> >         >
> >         > Caused by: org.apache.nifi.controller.UninheritableFlowException:
> Failed to
> >         > connect node to cluster because local flow is different than
> cluster flow.
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.
> loadFromConnectionResponse(StandardFlowService.java:879)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.load(
> StandardFlowService.java:493)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         ... 3 common frames omitted
> >         >
> >         > Caused by: org.apache.nifi.controller.UninheritableFlowException:
> Proposed
> >         > Authorizer is not inheritable by the flow controller because
> of Authorizer
> >         > differences: Proposed Authorizations do not match current
> Authorizations
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowSynchronizer.sync(
> StandardFlowSynchronizer.java:252)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.FlowController.synchronize(
> FlowController.java:1435)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.persistence.StandardXMLFlowConfigurationDA
> O.load(StandardXMLFlowConfigurationDAO.java:83)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.loadFromBytes(
> StandardFlowService.java:671)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         at
> >         > org.apache.nifi.controller.StandardFlowService.
> loadFromConnectionResponse(StandardFlowService.java:857)
> >         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
> >         >
> >         >         ... 4 common frames omitted
> >         >
> >         > [root@ncm-cm1 logs]#
> >         >
> >         >
> >         >
> >         > I don’t know if the ‘Proposed Authorizer is not inheritable…’
> exception is
> >         > part of the problem too.
> >         >
> >         > The docs weren’t very clear on whether (when upgrading and
> using the legacy
> >         > support of the authorized-user.xml path required the nodes to
> be also added
> >         > to the authorizers.xml.
> >         >
> >         > I did add them in the end as various attempts to get the
> cluster up and
> >         > running without them failed (as each server didn’t seem to
> have rights to do
> >         > anything.
> >         >
> >         >
> >         >
> >         > I have a lot of RPG in my work flows as I am ingesting many
> syslog data
> >         > sources and this was the recommended pattern to distribute the
> data
> >         > (listensyslog…run on primary, output to port (RPG), pick up in
> rest of data
> >         > flow),
> >         >
> >         >
> >         >
> >         > Any suggestions on where to start trying to get this working?
> >         >
> >         > I’ve tried creating a new RPG on one on the datanodes and
> connecting the
> >         > syslog to that which sort of worked but then I have a bunch of
> other errors
> >         > when trying to enable the ports to do with not being able to
> connect to
> >         > (what was) the NCM.
> >         >
> >         >
> >         >
> >         > Thanks
> >         >
> >         > Conrad
> >         >
> >         >
> >         >
> >         > SecureData, combating cyber threats
> >         >
> >         > ________________________________
> >         >
> >         > The information contained in this message or any of its
> attachments may be
> >         > privileged and confidential and intended for the exclusive use
> of the
> >         > intended recipient. If you are not the intended recipient any
> disclosure,
> >         > reproduction, distribution or other dissemination or use of
> this
> >         > communications is strictly prohibited. The views expressed in
> this email are
> >         > those of the individual and not necessarily of SecureData
> Europe Ltd. Any
> >         > prices quoted are only valid if followed up by a formal
> written quote.
> >         >
> >         > SecureData Europe Limited. Registered in England & Wales
> 04365896.
> >         > Registered Address: SecureData House, Hermitage Court,
> Hermitage Lane,
> >         > Maidstone, Kent, ME16 9NT
> >
> >
> >          ***This email originated outside SecureData***
> >
> >         Click https://www.mailcontrol.com/sr/tAj77!!
> uP0XGX2PQPOmvUu5zZAYN1Mos55ZMH65vS49VoLnJlQAkvDtaSciXa9lO25L
> WvxYjTGeVGm43FW9a3A==
> <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==>  to report this
> email as spam.
> >
> >
> >
> >
>
>
>
>
>

Mime
View raw message