Return-Path: X-Original-To: apmail-nifi-users-archive@minotaur.apache.org Delivered-To: apmail-nifi-users-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 626B9182A7 for ; Tue, 29 Sep 2015 23:27:38 +0000 (UTC) Received: (qmail 35551 invoked by uid 500); 29 Sep 2015 23:27:38 -0000 Delivered-To: apmail-nifi-users-archive@nifi.apache.org Received: (qmail 35524 invoked by uid 500); 29 Sep 2015 23:27:38 -0000 Mailing-List: contact users-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@nifi.apache.org Delivered-To: mailing list users@nifi.apache.org Received: (qmail 35514 invoked by uid 99); 29 Sep 2015 23:27:38 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Sep 2015 23:27:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C66C1C0479 for ; Tue, 29 Sep 2015 23:27:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.9 X-Spam-Level: ** X-Spam-Status: No, score=2.9 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id beImOLhufCDf for ; Tue, 29 Sep 2015 23:27:25 +0000 (UTC) Received: from mail-qg0-f45.google.com (mail-qg0-f45.google.com [209.85.192.45]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id AE52142B5D for ; Tue, 29 Sep 2015 23:27:25 +0000 (UTC) Received: by qgev79 with SMTP id v79so20907492qge.0 for ; Tue, 29 Sep 2015 16:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=8BU4L6C1jO2BmOj9XD7L4EydEG4LQJjpeXcDewYnDas=; b=rP2/jULus6r7X3HEXWIZUzMW5tDVo4k4bnOwqBEjMfP9uN70zU3/+6lqN7zmcNWXgt XJiF+nmamYId4dpyIyDh37hFERNX6pzmn4jj3mFBIPBljDcI1gw3DWBpeStHZV1YXR6S 9X9qUagXRo+TcAdvKnh+lzSaDoQshTza8QcI3X/qbj2xgNZkrCjylQgQxC5K0cmYlOUz LuMU6dFXI9E7WSk1tzYGVJdlmusnK0pU8+MG7yFXmrqhwDPIFVpjBN/g+QoU9wB6+PtK 12/Mz1CYQBnWEHm56X870ImgiDAQC4iMirUHWQ0kI/OgFd0j0T/lNXHWA3pQ6DkRJoFW hNGQ== X-Received: by 10.140.89.47 with SMTP id u44mr704532qgd.67.1443569245337; Tue, 29 Sep 2015 16:27:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.29.137 with HTTP; Tue, 29 Sep 2015 16:26:45 -0700 (PDT) In-Reply-To: References: From: Aldrin Piri Date: Tue, 29 Sep 2015 19:26:45 -0400 Message-ID: Subject: Re: nifi Cluster setup issue To: users@nifi.apache.org Content-Type: multipart/alternative; boundary=001a11c133d613deb30520eb2800 --001a11c133d613deb30520eb2800 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Chakrader, I suspect that the nifi.web.http.host property is not using the same address as that specified and is transmitting "localhost" (the system's response to a localhost hostname lookup from Java). While the clustering protocol communicates via the properties you list, the actual command-control and replication of requests from master to slave nodes is carried out via the REST API which also runs on the web tier. The system's hostname, as previously determined, is transmitted as part of the clustering handshake. Either the system needs to have it report a valid hostname or a host needs to be specified for nifi.web.http.host. In either case of hostname or specified host, each must be network reachable from the master and able to be bound to locally within your server. Let us know if you need additional direction and we'd be happy to help you through the process. Thanks! On Tue, Sep 29, 2015 at 6:56 PM, Chakrader Dewaragatla < Chakrader.Dewaragatla@lifelock.com> wrote: > Hi =E2=80=93 We are exploring nifi for our workflow management, I have a = cluster > setup with 3 nodes. One as master and rest as slaves. > > I see following error when I try to access the nifi workflow webpage. > > 2015-09-29 22:46:13,263 WARN [NiFi Web Server-23] > o.a.n.c.m.impl.HttpRequestReplicatorImpl Node request for > [id=3D7481fca5-930c-4d4b-84a3-66cc62b4e2d3, apiAddress=3Dlocalhost, > apiPort=3D8080, socketAddress=3Dlocalhost, socketPort=3D3002] encountered > exception: java.util.concurrent.ExecutionException: > com.sun.jersey.api.client.ClientHandlerException: > java.net.ConnectException: Connection refused > > 2015-09-29 22:46:13,263 WARN [NiFi Web Server-23] > o.a.n.c.m.impl.HttpRequestReplicatorImpl Node request for > [id=3D0abd8295-34a3-4bf7-ab06-1b6b94014740, apiAddress=3Dlocalhost, > apiPort=3D8080, socketAddress=3D10.233.2.42, socketPort=3D3002] encounter= ed > exception: java.util.concurrent.ExecutionException: > com.sun.jersey.api.client.ClientHandlerException: > java.net.ConnectException: Connection refused > > 2015-09-29 22:46:13,264 INFO [NiFi Web Server-23] > o.a.n.c.m.e.NoConnectedNodesException > org.apache.nifi.cluster.manager.exception.NoResponseFromNodesException: N= o > nodes were able to process this request.. Returning Conflict response. > > > Master is not hybrid, I wonder why it is trying to self connect 3002. > > > Master settings: > > # cluster manager properties (only configure for cluster manager) # > > nifi.cluster.is.manager=3Dtrue > > nifi.cluster.manager.address=3D10.233.2.40 > > nifi.cluster.manager.protocol.port=3D3001 > > nifi.cluster.manager.node.firewall.file=3D > > nifi.cluster.manager.node.event.history.size=3D10 > > nifi.cluster.manager.node.api.connection.timeout=3D30 sec > > nifi.cluster.manager.node.api.read.timeout=3D30 sec > > nifi.cluster.manager.node.api.request.threads=3D10 > > nifi.cluster.manager.flow.retrieval.delay=3D5 sec > > nifi.cluster.manager.protocol.threads=3D10 > > nifi.cluster.manager.safemode.duration=3D0 sec > > > Slave settings: > > # cluster node properties (only configure for cluster nodes) # > > nifi.cluster.is.node=3Dtrue > > nifi.cluster.node.address=3D10.233.2.42 > > nifi.cluster.node.protocol.port=3D3002 > > nifi.cluster.node.protocol.threads=3D2 > > # if multicast is not used, nifi.cluster.node.unicast.xxx must have same > values as nifi.cluster.manager.xxx # > > nifi.cluster.node.unicast.manager.address=3D10.233.2.40 > > nifi.cluster.node.unicast.manager.protocol.port=3D3001 > > > > > ------------------------------ > The information contained in this transmission may contain privileged and > confidential information. It is intended only for the use of the person(s= ) > named above. If you are not the intended recipient, you are hereby notifi= ed > that any review, dissemination, distribution or duplication of this > communication is strictly prohibited. If you are not the intended > recipient, please contact the sender by reply email and destroy all copie= s > of the original message. > ------------------------------ > --001a11c133d613deb30520eb2800 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Chakrader,

I suspect that the nifi.web.= http.host property is not using the same address as that specified and is t= ransmitting "localhost" (the system's response to a localhost= hostname lookup from Java).=C2=A0 While the clustering protocol communicat= es via the properties you list, the actual command-control and replication = of requests from master to slave nodes is carried out via the REST API whic= h also runs on the web tier.=C2=A0 The system's hostname, as previously= determined, is transmitted as part of the clustering handshake.
=
Either the system needs to have it report a valid hostname o= r a host needs to be specified for=C2=A0nifi.web.http.host.=C2=A0 In either= case of hostname or specified host, each must be network reachable from th= e master and able to be bound to locally within your server.

=
Let us know if you need additional direction and we'd be hap= py to help you through the process.

Thanks!
<= /div>

On Tue, Sep = 29, 2015 at 6:56 PM, Chakrader Dewaragatla <Chakrader.Dew= aragatla@lifelock.com> wrote:
Hi =E2=80=93 We are exploring nifi for our workflow management, I have= a cluster setup with 3 nodes. One as master and rest as slaves.

I see following error when I try to access the nifi workflow webpage.= =C2=A0

2015-09-29 22:46:1= 3,263 WARN [NiFi Web Server-23] o.a.n.c.m.impl.HttpRequestReplicatorImpl No= de request for [id=3D7481fca5-930c-4d4b-84a3-66cc62b4e2d3, apiAddress=3Dloc= alhost, apiPort=3D8080, socketAddress=3Dlocalhost, socketPort=3D3002] encountered exception: java.util.concurrent.ExecutionEx= ception: com.sun.jersey.api.client.ClientHandlerException: java.net.Connect= Exception: Connection refused

2015-09-29 22:46:1= 3,263 WARN [NiFi Web Server-23] o.a.n.c.m.impl.HttpRequestReplicatorImpl No= de request for [id=3D0abd8295-34a3-4bf7-ab06-1b6b94014740, apiAddress=3Dloc= alhost, apiPort=3D8080, socketAddress=3D10.233.2.42, socketPort=3D3002] encountered exception: java.util.concurrent.ExecutionEx= ception: com.sun.jersey.api.client.ClientHandlerException: java.net.Connect= Exception: Connection refused

2015-09-29 22:46:1= 3,264 INFO [NiFi Web Server-23] o.a.n.c.m.e.NoConnectedNodesException org.a= pache.nifi.cluster.manager.exception.NoResponseFromNodesException: No nodes= were able to process this request.. Returning Conflict response.


Master is not hybr= id, I wonder why it is trying to self connect 3002.


Master settings:= =C2=A0

# cluster manager = properties (only configure for cluster manager) #

nifi.cluster.is.ma= nager=3Dtrue

nifi.cluster.manag= er.address=3D10.233.2.40

nifi.cluster.manag= er.protocol.port=3D3001

nifi.cluster.manag= er.node.firewall.file=3D

nifi.cluster.manag= er.node.event.history.size=3D10

nifi.cluster.manag= er.node.api.connection.timeout=3D30 sec

nifi.cluster.manag= er.node.api.read.timeout=3D30 sec

nifi.cluster.manag= er.node.api.request.threads=3D10

nifi.cluster.manag= er.flow.retrieval.delay=3D5 sec

nifi.cluster.manag= er.protocol.threads=3D10

nifi.cluster.manag= er.safemode.duration=3D0 sec


Slave settings:=C2= =A0

# cluster node pro= perties (only configure for cluster nodes) #

nifi.cluster.is.no= de=3Dtrue

nifi.cluster.node.= address=3D10.233.2.42

nifi.cluster.node.= protocol.port=3D3002

nifi.cluster.node.= protocol.threads=3D2

# if multicast is = not used, nifi.cluster.node.unicast.xxx must have same values as nifi.clust= er.manager.xxx #

nifi.cluster.node.= unicast.manager.address=3D10.233.2.40

nifi.cluster.node.= unicast.manager.protocol.port=3D3001




The information contained in this transmission may contain privileged and c= onfidential information. It is intended only for the use of the person(s) n= amed above. If you are not the intended recipient, you are hereby notified = that any review, dissemination, distribution or duplication of this communication is strictly prohibited. = If you are not the intended recipient, please contact the sender by reply e= mail and destroy all copies of the original message.

--001a11c133d613deb30520eb2800--