nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knapp, Michael" <Michael.Kn...@capitalone.com>
Subject host names in a clustered NiFi
Date Tue, 08 Aug 2017 15:09:29 GMT
Hi,

I’m trying to run NiFi in a clustered configuration within docker containers on kubernetes.
 While I have NiFi starting in standalone mode, I get exceptions when launching it in clustered
mode.  I’m having a lot of trouble figuring out what the host names really should be.

I looked through the system properties<http://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system_properties>
(web and cluster properties) and also the Admin Guide (Cluster setup section).  The description
of these host properties is quite vague:

                nifi.cluster.node.address – The fully qualified address of the node.  It
is blank by default.
                Nifi.web.http.host – The HTTP host.  It is blank by default.
                Nifi.web.https.host – The HTTPS host. It is blank by default.

The descriptions above are quite vague and don’t explain to me how NiFi will behave based
on these values.  In fact, I keep needing to go through a lot of trial and error to make this
work.  What happens if they are blank?  What happens if I insert the IP address?

Just asking for the host name is ambiguous, I have a docker internal host name, an EC2 host
name, a kubernetes service name, load balancer host names, and a route53 host name.  The correct
value really depends on how these are used in the code.  I could easily see these needing
to be the kubernetes service name or the internal host name, or the route53 host name.  What
should I be using here?  Should all nodes in the cluster have the same value for this?

Here is the stack trace I am seeing at runtime:

2017-08-08 14:23:51,534 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start
web server: Unresolved address
2017-08-08 14:23:51,535 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut 2017-08-08 14:23:51,536
WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting
down.
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut java.net.SocketException:
Unresolved address
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.Net.translateToSocketException(Net.java:131)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.Net.translateException(Net.java:157)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.Net.translateException(Net.java:163)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:76)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:298)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.server.Server.doStart(Server.java:431)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.apache.nifi.NiFi.<init>(NiFi.java:160)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at org.apache.nifi.NiFi.main(NiFi.java:267)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut Caused by: java.nio.channels.UnresolvedAddressException:
null
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.Net.checkAddress(Net.java:101)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:218)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
2017-08-08 14:23:51,536 INFO [NiFi logging handler] org.apache.nifi.StdOut ... 9 common frames
omitted


Michael Knapp
________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates and may only be used solely in performance of work or services for Capital
One. The information transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended recipient, you
are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message