Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 108BF200BC3 for ; Fri, 18 Nov 2016 16:41:58 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 0EF78160B04; Fri, 18 Nov 2016 15:41:58 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2F784160B03 for ; Fri, 18 Nov 2016 16:41:57 +0100 (CET) Received: (qmail 2568 invoked by uid 500); 18 Nov 2016 15:41:56 -0000 Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list dev@nifi.apache.org Received: (qmail 2552 invoked by uid 99); 18 Nov 2016 15:41:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Nov 2016 15:41:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 06AB8C0FD4 for ; Fri, 18 Nov 2016 15:41:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 8cw0k8POOBdG for ; Fri, 18 Nov 2016 15:41:52 +0000 (UTC) Received: from mail-qk0-f182.google.com (mail-qk0-f182.google.com [209.85.220.182]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id CD76E5F1E5 for ; Fri, 18 Nov 2016 15:41:51 +0000 (UTC) Received: by mail-qk0-f182.google.com with SMTP id x190so267474064qkb.0 for ; Fri, 18 Nov 2016 07:41:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=YHQsUtAFv8QdDyEFWRmnG9TeAs7L0/YPTQxrUK7/p/A=; b=REuKcjmG+seZizfnfQL1Wducqu74Sf6+wUwJoYYzwmKYhS50vVpjaTqaMpK7H+omVI b33kVjjEUajGN2yu3r2/0ArVQV/CvfrkY68L01cebeXyU9Nn3eLBd2gLQ8ISaisK7uNc W3fiXLSXDRKNWS6FzRpyqVwOUC9xhR66/nayY0ax0g9U3LZkWShjvER7w/sjJ0Wp+9ck h7d469j1SKAhSjvee0YE2+KjAjduLbv7J6xRBPQMPVkQ4oThmyP52eilQ5SSiwqY0yF8 58rms78nB/LTClWlE9NwDI1cSgVZykURUqAZLUtqtBgj2muWpYDTKaG0s0ePftrHRlJi Vz0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=YHQsUtAFv8QdDyEFWRmnG9TeAs7L0/YPTQxrUK7/p/A=; b=QFj/fSi2YSy/NejOryIFqpDfDzFKZ+l4Njry001TQB/6gBMMniBtmkMZXpzolBVhzE I05hkYeYt7FH1F/cVHRXh6PObeOar6CuqfZNhS6YjTZaLahHXACtQSuHewxcNrbft/wf ZTWUC11eOW7y8gcz9gkZuzh1L6ZCDHnusOU6/1a41gCaXrOVXAVc32o+maV0+Jcw6qKe BmIhYfLwiqhlUMYJkOm2vUfqyvesJkDW2kQpRqhmIk5ZpQIrv/cUL43/+oX9uAPP2pvI vxUFYFoFW8VcezgbCpRmXqQO80JMV1OUWaYF5+HLi/l1J0UIVnZlxEl6syTkRl3Dugw1 ri9w== X-Gm-Message-State: AKaTC00fjPbQIB3gI/hGl8aZLvzu4W9LmxcS7Tls7aB/6U7GuK/96DH2/0HO9R5QGcogaAtbg11ACsfQdd9JWg== X-Received: by 10.55.183.3 with SMTP id h3mr420809qkf.33.1479483710770; Fri, 18 Nov 2016 07:41:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.140.130 with HTTP; Fri, 18 Nov 2016 07:41:49 -0800 (PST) In-Reply-To: References: From: Joe Gresock Date: Fri, 18 Nov 2016 15:41:49 +0000 Message-ID: Subject: Re: Zookeeper error To: dev@nifi.apache.org Content-Type: multipart/alternative; boundary=94eb2c07106a08183905419525c3 archived-at: Fri, 18 Nov 2016 15:41:58 -0000 --94eb2c07106a08183905419525c3 Content-Type: text/plain; charset=UTF-8 It appears that if you try to start up just one node in a cluster with multiple zk hosts specified in zookeeper.properties, you get this error spammed at an incredible rate in your logs. When I started up all 3 nodes at once, they didn't receive the error. On Fri, Nov 18, 2016 at 3:18 PM, Joe Gresock wrote: > I'm upgrading a test 0.x nifi cluster to 1.x using the latest in master as > of today. > > I was able to successfully start the 3-node cluster once, but then I > restarted it and get the following error spammed in the nifi-app.log. > > I'm not sure where to start debugging this, and I'm puzzled why it would > work once and then start giving me errors on the second restart. Has > anyone run into this error? > > 2016-11-18 15:07:18,178 INFO [main] org.eclipse.jetty.server.Server > Started @83426ms > 2016-11-18 15:07:18,883 INFO [main] org.apache.nifi.web.server.JettyServer > Loading Flow... > 2016-11-18 15:07:18,889 INFO [main] org.apache.nifi.io.socket.SocketListener > Now listening for connections from nodes on port 9001 > 2016-11-18 15:07:19,117 INFO [main] o.a.nifi.controller.StandardFlowService > Connecting Node: ip-172-31-33-34.ec2.internal:8443 > 2016-11-18 15:07:25,781 WARN [main] o.a.nifi.controller.StandardFlowService > There is currently no Cluster Coordinator. This often happens upon restart > of NiFi when running an embedded ZooKeeper. Will register this node to > become the active Cluster Coordinator and will attempt to connect to > cluster again > 2016-11-18 15:07:25,782 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager > CuratorLeaderElectionManager[stopped=false] Attempted to register Leader > Election for role 'Cluster Coordinator' but this role is already registered > 2016-11-18 15:07:34,685 WARN [main] o.a.nifi.controller.StandardFlowService > There is currently no Cluster Coordinator. This often happens upon restart > of NiFi when running an embedded ZooKeeper. Will register this node to > become the active Cluster Coordinator and will attempt to connect to > cluster again > 2016-11-18 15:07:34,685 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager > CuratorLeaderElectionManager[stopped=false] Attempted to register Leader > Election for role 'Cluster Coordinator' but this role is already registered > 2016-11-18 15:07:34,696 INFO [Curator-Framework-0] o.a.c.f.state.ConnectionStateManager > State change: SUSPENDED > 2016-11-18 15:07:34,698 INFO [Curator-ConnectionStateManager-0] > o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller. > leader.election.CuratorLeaderElectionManager$ElectionListener@671a652a > Connection State changed to SUSPENDED > > *2016-11-18 15:07:34,699 ERROR [Curator-Framework-0] > o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave > uporg.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss* > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > ~[zookeeper-3.4.6.jar:3.4.6-1569965] > at org.apache.curator.framework.imps.CuratorFrameworkImpl. > checkBackgroundRetry(CuratorFrameworkImpl.java:728) > [curator-framework-2.11.0.jar:na] > at org.apache.curator.framework.imps.CuratorFrameworkImpl. > performBackgroundOperation(CuratorFrameworkImpl.java:857) > [curator-framework-2.11.0.jar:na] > at org.apache.curator.framework.imps.CuratorFrameworkImpl. > backgroundOperationsLoop(CuratorFrameworkImpl.java:809) > [curator-framework-2.11.0.jar:na] > at org.apache.curator.framework.imps.CuratorFrameworkImpl. > access$300(CuratorFrameworkImpl.java:64) [curator-framework-2.11.0.jar:na] > at org.apache.curator.framework.imps.CuratorFrameworkImpl$4. > call(CuratorFrameworkImpl.java:267) [curator-framework-2.11.0.jar:na] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_111] > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_111] > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.8.0_111] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_111] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_111] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111] > > > -- > I know what it is to be in need, and I know what it is to have plenty. I > have learned the secret of being content in any and every situation, > whether well fed or hungry, whether living in plenty or in want. I can > do all this through him who gives me strength. *-Philippians 4:12-13* > -- I know what it is to be in need, and I know what it is to have plenty. I have learned the secret of being content in any and every situation, whether well fed or hungry, whether living in plenty or in want. I can do all this through him who gives me strength. *-Philippians 4:12-13* --94eb2c07106a08183905419525c3--