nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: Fixing unstable nifi cluster.
Date Tue, 16 Oct 2018 12:03:01 GMT
Hi,

Can you try increasing the below parameters? That's usually what I
recommend, our default values being probably a bit too aggressively low.

nifi.zookeeper.connect.timeout=15 secs
nifi.zookeeper.session.timeout=15 secs
nifi.cluster.node.read.timeout=30 sec

Pierre

Le mar. 16 oct. 2018 à 13:02, ashwin konale <ashwin.konale@gmail.com> a
écrit :

> Hi,
> We have a 3 node nifi cluster (With separate zookeper instances running in
> the same machines) which pulls the data from mysql and write to hdfs. I am
> frequently running into problems with cluster. Nodes keeps disconnecting
> from each other, primary nodes keeps switching and sometimes it just goes
> into zombie state when I just cannot access the ui. I have followed best
> practices guide and tweaked params in nifi.properties, have switched
> provenanceRepositoryImplementation to volatile because cluster was not able
> to keep up with incoming traffic. Data traffic is not high at all (4Mbps).
> This is the message I frequently get from the logs.
>
> *INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change:
> LOST*
> *INFO [Curator-ConnectionStateManager-0]
> o.a.n.c.l.e.CuratorLeaderElectionManager
>
> org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@56ebedec
> Connection State changed to LOST*
> *INFO [Curator-ConnectionStateManager-0]
> o.a.n.c.l.e.CuratorLeaderElectionManager
>
> org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@1b0e2055
> Connection State changed to LOST*
> *INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change:
> RECONNECTED*
>
> Am I doing something wrong with cluster setup ? Can someone give me some
> guidance on how to go about debugging this issue ? What kind of system
> metrics to look at etc.
>
> Ashwin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message