nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Conrad Crampton <conrad.cramp...@SecData.com>
Subject Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups
Date Wed, 19 Oct 2016 15:52:03 GMT
Hi,
The nifi-user.log doesn’t show any errors – in fact it shows success for any authentication
to the old NCM server. What is odd though is the old NCM server is the only one out of the
7 servers that I can’t log into at
https://xxxx:9090/nifi where I can with all the others on their respective ports and hostnames.

I’ll give SSL debug a go, but as a plan for tomorrow – I have generated new keystores,
truststores, client certts  etc. for all nodes in my cluster using the nifi-toolkit.
Would it be worth using all these newly created ones or will it break existing flowfiles and
data held in queues etc.?

Thanks for your help so far.
Conrad


From: Bryan Bende <bbende@gmail.com>
Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
Date: Wednesday, 19 October 2016 at 16:38
To: "users@nifi.apache.org" <users@nifi.apache.org>
Subject: Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

Yes http site-to-site was added recently so setting that to disabled should be fine and not
related.

If you are using all the same keystores and truststores from before, then I can't think of
why the nodes wouldn't be able to communicate securely.

Unless anyone else has some other ideas, you may need to turn on SSL debug (-Djavax.net.debug=all)
to see why the handshake is failing.

Is there anything interesting/related in nifi-user.log?

On Wed, Oct 19, 2016 at 10:38 AM, Conrad Crampton <conrad.crampton@secdata.com<mailto:conrad.crampton@secdata.com>>
wrote:
Hi,
Yes, every nifi.properties is set thus – with host and port different for each.

# Site to Site properties
nifi.remote.input.socket.host=ncm.xxxxxxx
nifi.remote.input.socket.port=9870
nifi.remote.input.secure=true
nifi.remote.input.http.enabled=false
nifi.remote.input.http.transaction.ttl=30 sec

You’ll obviously notice that I have http disabled. I set this as this was a new setting
which I didn’t have before (it was only RAW in previous versions wasn’t it?)

Does this make a difference?

Thanks
Conrad

From: Bryan Bende <bbende@gmail.com<mailto:bbende@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Wednesday, 19 October 2016 at 15:33

To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

Trying to think of things to check here...

Does every node have nifi.remote.input.secure=true in nifi.properties and the URL in the RPG
is an https URL?

On Wed, Oct 19, 2016 at 10:25 AM, Conrad Crampton <conrad.crampton@secdata.com<mailto:conrad.crampton@secdata.com>>
wrote:
One other thing…
The RPGs have an unlocked padlock on them saying S2S is not secure.
Conrad

From: Bryan Bende <bbende@gmail.com<mailto:bbende@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Wednesday, 19 October 2016 at 15:20
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

Ok that does seem like a TLS/SSL issue...

Is this a single cluster doing site-to-site to itself?

On Wed, Oct 19, 2016 at 10:06 AM, Joe Witt <joe.witt@gmail.com<mailto:joe.witt@gmail.com>>
wrote:
thanks conrad - did get it.  Bryan is being more helpful that I so I
went silent :-)

On Wed, Oct 19, 2016 at 10:02 AM, Conrad Crampton
<conrad.crampton@secdata.com<mailto:conrad.crampton@secdata.com>> wrote:
> Hi Joe,
>     Yep,
>     Tried removing the RPG that referenced the NCM and adding new one with one of the
datanodes as url.
>     That sort of worked, but kept getting errors about the NCM not being available for
the ports and therefore couldn’t actually enable the port I needed to for that RPG.
>     Thanks
>     Conrad
>
> (sending again as don’t know if the stupid header ‘spoofed’ is stopping getting
though – apologies if already sent)
>
>     On 19/10/2016, 14:12, "Joe Witt" <joe.witt@gmail.com<mailto:joe.witt@gmail.com>>
wrote:
>
>         Conrad,
>
>         For s2s now you can just point at any of the nodes in the cluster.
>         Have you tried changing the URL or removing and adding new RPG
>         entries?
>
>         Thanks
>         Joe
>
>         On Wed, Oct 19, 2016 at 8:38 AM, Conrad Crampton
>         <conrad.crampton@secdata.com<mailto:conrad.crampton@secdata.com>>
wrote:
>         > Hi,
>         >
>         > I have finally taken the plunge to upgrade my cluster from 0.6.1 to 1.0.0.
>         >
>         > 6 nodes with a NCM.
>         >
>         > With the removal of NCM in 1.0.0 I believe I now have an issue where none
of
>         > my Remote Process Groups work as they previously did because they were
>         > configured to connect to the NCM (as the RPG url) which now doesn’t exist.
>         >
>         > I have tried converting my NCM to a node but whilst I can get it running
>         > (sort of) when I try and connect to the cluster I get something like this
in
>         > my logs…
>         >
>         >
>         >
>         > 2016-10-19 13:14:44,109 ERROR [main] o.a.nifi.controller.StandardFlowService
>         > Failed to load flow from cluster due to:
>         > org.apache.nifi.controller.UninheritableFlowException: Failed to connect
>         > node to cluster because local flow is different than cluster flow.
>         >
>         > org.apache.nifi.controller.UninheritableFlowException: Failed to connect
>         > node to cluster because local flow is different than cluster flow.
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746)
>         > [nifi-jetty-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         > Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed
>         > Authorizer is not inheritable by the flow controller because of Authorizer
>         > differences: Proposed Authorizations do not match current Authorizations
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         ... 4 common frames omitted
>         >
>         > 2016-10-19 13:14:44,414 ERROR [main] o.a.n.c.c.node.NodeClusterCoordinator
>         > Event Reported for ncm-cm1.mis-cds.local:9090 -- Node disconnected from
>         > cluster due to org.apache.nifi.controller.UninheritableFlowException: Failed
>         > to connect node to cluster because local flow is different than cluster
>         > flow.
>         >
>         > 2016-10-19 13:14:44,420 ERROR [Shutdown Cluster Coordinator]
>         > org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[Shutdown
>         > Cluster Coordinator,5,main]: java.lang.NullPointerException
>         >
>         > 2016-10-19 13:14:44,423 ERROR [Shutdown Cluster Coordinator]
>         > org.apache.nifi.NiFi
>         >
>         > java.lang.NullPointerException: null
>         >
>         >         at
>         > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
>         > ~[na:1.8.0_51]
>         >
>         >         at
>         > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
>         > ~[na:1.8.0_51]
>         >
>         >         at
>         > org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.updateNodeStatus(NodeClusterCoordinator.java:570)
>         > ~[nifi-framework-cluster-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.shutdown(NodeClusterCoordinator.java:119)
>         > ~[nifi-framework-cluster-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService$1.run(StandardFlowService.java:330)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_51]
>         >
>         > 2016-10-19 13:14:44,448 WARN [main] o.a.n.c.l.e.CuratorLeaderElectionManager
>         > Failed to close Leader Selector for Cluster Coordinator
>         >
>         > java.lang.IllegalStateException: Already closed or has not been started
>         >
>         >         at
>         > com.google.common.base.Preconditions.checkState(Preconditions.java:173)
>         > ~[guava-18.0.jar:na]
>         >
>         >         at
>         > org.apache.curator.framework.recipes.leader.LeaderSelector.close(LeaderSelector.java:270)
>         > ~[curator-recipes-2.11.0.jar:na]
>         >
>         >         at
>         > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.stop(CuratorLeaderElectionManager.java:159)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.FlowController.shutdown(FlowController.java:1303)
>         > [nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.stop(StandardFlowService.java:339)
>         > [nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:753)
>         > [nifi-jetty-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         > 2016-10-19 13:14:45,062 WARN [Cluster Socket Listener]
>         > org.apache.nifi.io.socket.SocketListener Failed to communicate with Unknown
>         > Host due to java.net.SocketException: Socket closed
>         >
>         > java.net.SocketException: Socket closed
>         >
>         >         at java.net.PlainSocketImpl.socketAccept(Native Method)
>         > ~[na:1.8.0_51]
>         >
>         >         at
>         > java.net<http://java.net>.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
>         > ~[na:1.8.0_51]
>         >
>         >         at java.net.ServerSocket.implAccept(ServerSocket.java:545)
>         > ~[na:1.8.0_51]
>         >
>         >         at
>         > sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:348)
>         > ~[na:1.8.0_51]
>         >
>         >         at
>         > org.apache.nifi.io.socket.SocketListener$2.run(SocketListener.java:112)
>         > ~[nifi-socket-utils-1.0.0.jar:1.0.0]
>         >
>         >         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
>         >
>         > 2016-10-19 13:14:45,064 WARN [main] org.apache.nifi.web.server.JettyServer
>         > Failed to start web server... shutting down.
>         >
>         > java.lang.Exception: Unable to load flow due to: java.io.IOException:
>         > org.apache.nifi.controller.UninheritableFlowException: Failed to connect
>         > node to cluster because local flow is different than cluster flow.
>         >
>         >         at
>         > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:755)
>         > ~[nifi-jetty-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.<init>(NiFi.java:152)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         >         at org.apache.nifi.NiFi.main(NiFi.java:243)
>         > [nifi-runtime-1.0.0.jar:1.0.0]
>         >
>         > Caused by: java.io.IOException:
>         > org.apache.nifi.controller.UninheritableFlowException: Failed to connect
>         > node to cluster because local flow is different than cluster flow.
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:497)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746)
>         > ~[nifi-jetty-1.0.0.jar:1.0.0]
>         >
>         >         ... 2 common frames omitted
>         >
>         > Caused by: org.apache.nifi.controller.UninheritableFlowException: Failed
to
>         > connect node to cluster because local flow is different than cluster flow.
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         ... 3 common frames omitted
>         >
>         > Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed
>         > Authorizer is not inheritable by the flow controller because of Authorizer
>         > differences: Proposed Authorizations do not match current Authorizations
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         at
>         > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857)
>         > ~[nifi-framework-core-1.0.0.jar:1.0.0]
>         >
>         >         ... 4 common frames omitted
>         >
>         > [root@ncm-cm1 logs]#
>         >
>         >
>         >
>         > I don’t know if the ‘Proposed Authorizer is not inheritable…’ exception
is
>         > part of the problem too.
>         >
>         > The docs weren’t very clear on whether (when upgrading and using the legacy
>         > support of the authorized-user.xml path required the nodes to be also added
>         > to the authorizers.xml.
>         >
>         > I did add them in the end as various attempts to get the cluster up and
>         > running without them failed (as each server didn’t seem to have rights
to do
>         > anything.
>         >
>         >
>         >
>         > I have a lot of RPG in my work flows as I am ingesting many syslog data
>         > sources and this was the recommended pattern to distribute the data
>         > (listensyslog…run on primary, output to port (RPG), pick up in rest of
data
>         > flow),
>         >
>         >
>         >
>         > Any suggestions on where to start trying to get this working?
>         >
>         > I’ve tried creating a new RPG on one on the datanodes and connecting the
>         > syslog to that which sort of worked but then I have a bunch of other errors
>         > when trying to enable the ports to do with not being able to connect to
>         > (what was) the NCM.
>         >
>         >
>         >
>         > Thanks
>         >
>         > Conrad
>         >
>         >
>         >
>         > SecureData, combating cyber threats
>         >
>         > ________________________________
>         >
>         > The information contained in this message or any of its attachments may
be
>         > privileged and confidential and intended for the exclusive use of the
>         > intended recipient. If you are not the intended recipient any disclosure,
>         > reproduction, distribution or other dissemination or use of this
>         > communications is strictly prohibited. The views expressed in this email
are
>         > those of the individual and not necessarily of SecureData Europe Ltd. Any
>         > prices quoted are only valid if followed up by a formal written quote.
>         >
>         > SecureData Europe Limited. Registered in England & Wales 04365896.
>         > Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>         > Maidstone, Kent, ME16 9NT
>
>
>          ***This email originated outside SecureData***
>
>         Click https://www.mailcontrol.com/sr/tAj77!!uP0XGX2PQPOmvUu5zZAYN1Mos55ZMH65vS49VoLnJlQAkvDtaSciXa9lO25LWvxYjTGeVGm43FW9a3A==<https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==>
 to report this email as spam.
>
>
>
>



Mime
View raw message