nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank Thiele (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-4241) ListSFTP node in cluster is not coming up
Date Tue, 01 Aug 2017 04:46:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108371#comment-16108371
] 

Frank Thiele commented on NIFI-4241:
------------------------------------

The WARN message is gone -- work around seems to work.
But the exception still remains. As I found NIFI-2343 I will check how to fix the configuration
-> Found the probable issue in {{state-management.xml}}:
{code}
...
    <cluster-provider>
        <id>zk-provider</id>
        <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
        <property name="Connect String"></property>
        <property name="Root Node">/nifi</property>
        <property name="Session Timeout">10 seconds</property>
        <property name="Access Control">Open</property>
    </cluster-provider>
...
{code}
Adding the addresses probably will work. Checking...

> ListSFTP node in cluster is not coming up
> -----------------------------------------
>
>                 Key: NIFI-4241
>                 URL: https://issues.apache.org/jira/browse/NIFI-4241
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.3.0
>         Environment: Docker container within VM:
> docker@myvm5:~$ uname -a
> Linux myvm5 4.4.74-boot2docker #1 SMP Mon Jun 26 18:01:14 UTC 2017 x86_64 GNU/Linux
>            Reporter: Frank Thiele
>         Attachments: cluster.png, flow.xml, Overview.png
>
>
> I have setup a small cluster with 3 nodes nifi1:8080, nifi2:8081 and nifi3:8082.
> When I configure a ListSFTP processor and run it, the following exception comes up:
> {code}
> 2017-07-31 06:25:28,522 ERROR [StandardProcessScheduler Thread-6] org.apache.nifi.engine.FlowEngine
A flow controller task execution stopped abnormally
> java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> 	at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.reflect.InvocationTargetException: null
> 	at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1307)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	... 2 common frames omitted
> Caused by: java.io.IOException: Failed to obtain value from ZooKeeper for component with
ID 9716842f-015d-1000-0000-00007de55c99 with exception code CONNECTIONLOSS
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:420)
> 	at org.apache.nifi.controller.state.StandardStateManager.getState(StandardStateManager.java:63)
> 	at org.apache.nifi.processor.util.list.AbstractListProcessor.updateState(AbstractListProcessor.java:199)
> 	... 15 common frames omitted
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /nifi/components/9716842f-015d-1000-0000-00007de55c99
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184)
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:403)
> 	... 17 common frames omitted
> 2017-07-31 06:25:28,523 ERROR [StandardProcessScheduler Thread-5] o.a.nifi.processors.standard.ListSFTP
ListSFTP[id=9716842f-015d-1000-0000-00007de55c99] ListSFTP[id=9716842f-015d-1000-0000-00007de55c99]
failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing
one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds:
java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.
> java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.
> 	at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1482)
> 	at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> 	at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> 	at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> 	... 9 common frames omitted
> Caused by: java.lang.reflect.InvocationTargetException: null
> 	at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1307)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1303)
> 	... 6 common frames omitted
> Caused by: java.io.IOException: Failed to obtain value from ZooKeeper for component with
ID 9716842f-015d-1000-0000-00007de55c99 with exception code CONNECTIONLOSS
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:420)
> 	at org.apache.nifi.controller.state.StandardStateManager.getState(StandardStateManager.java:63)
> 	at org.apache.nifi.processor.util.list.AbstractListProcessor.updateState(AbstractListProcessor.java:199)
> 	... 15 common frames omitted
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /nifi/components/9716842f-015d-1000-0000-00007de55c99
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184)
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:403)
> 	... 17 common frames omitted
> 2017-07-31 06:25:28,523 ERROR [StandardProcessScheduler Thread-5] o.a.n.controller.StandardProcessorNode
Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing
one of processor's OnScheduled task.
> java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task.
> 	at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1482)
> 	at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> 	at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> 	at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> 	... 9 common frames omitted
> Caused by: java.lang.reflect.InvocationTargetException: null
> 	at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
> 	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1307)
> 	at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1303)
> 	... 6 common frames omitted
> Caused by: java.io.IOException: Failed to obtain value from ZooKeeper for component with
ID 9716842f-015d-1000-0000-00007de55c99 with exception code CONNECTIONLOSS
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:420)
> 	at org.apache.nifi.controller.state.StandardStateManager.getState(StandardStateManager.java:63)
> 	at org.apache.nifi.processor.util.list.AbstractListProcessor.updateState(AbstractListProcessor.java:199)
> 	... 15 common frames omitted
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
= ConnectionLoss for /nifi/components/9716842f-015d-1000-0000-00007de55c99
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184)
> 	at org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.getState(ZooKeeperStateProvider.java:403)
> 	... 17 common frames omitted
> {code}
> The only kind-of-related warning I can find is this:
> {code}
> 2017-07-31 06:25:05,763 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater
Heartbeat created at 2017-07-31 06:25:05,754 and sent to nifi1:10600 at 2017-07-31 06:25:05,763;
send took 9 millis
> 2017-07-31 06:25:07,947 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
Could not communicate with nifi1:8081 to determine which nodes exist in the remote NiFi cluster,
due to org.apache.http.conn.HttpHostConnectException: Connect to nifi1:8081 [nifi1/10.0.0.32]
failed: Connection refused (Connection refused)
> 2017-07-31 06:25:07,947 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@4516d212 Unable to refresh Remote Group's peers
due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist
in the remote cluster
> 2017-07-31 06:25:10,775 INFO [Clustering Tasks Thread-1] o.a.n.c.c.ClusterProtocolHeartbeater
Heartbeat created at 2017-07-31 06:25:10,764 and sent to nifi1:10600 at 2017-07-31 06:25:10,775;
send took 10 millis
> 2017-07-31 06:25:12,951 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
Could not communicate with nifi1:8081 to determine which nodes exist in the remote NiFi cluster,
due to org.apache.http.conn.HttpHostConnectException: Connect to nifi1:8081 [nifi1/10.0.0.32]
failed: Connection refused (Connection refused)
> 2017-07-31 06:25:12,951 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@4516d212 Unable to refresh Remote Group's peers
due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist
in the remote cluster
> 2017-07-31 06:25:15,781 INFO [Clustering Tasks Thread-3] o.a.n.c.c.ClusterProtocolHeartbeater
Heartbeat created at 2017-07-31 06:25:15,775 and sent to nifi1:10600 at 2017-07-31 06:25:15,781;
send took 5 millis
> 2017-07-31 06:25:17,954 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
Could not communicate with nifi1:8081 to determine which nodes exist in the remote NiFi cluster,
due to org.apache.http.conn.HttpHostConnectException: Connect to nifi1:8081 [nifi1/10.0.0.32]
failed: Connection refused (Connection refused)
> 2017-07-31 06:25:17,955 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@4516d212 Unable to refresh Remote Group's peers
due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist
in the remote cluster
> 2017-07-31 06:25:20,790 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater
Heartbeat created at 2017-07-31 06:25:20,782 and sent to nifi1:10600 at 2017-07-31 06:25:20,790;
send took 7 millis
> 2017-07-31 06:25:22,959 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
Could not communicate with nifi1:8081 to determine which nodes exist in the remote NiFi cluster,
due to org.apache.http.conn.HttpHostConnectException: Connect to nifi1:8081 [nifi1/10.0.0.32]
failed: Connection refused (Connection refused)
> 2017-07-31 06:25:22,959 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@4516d212 Unable to refresh Remote Group's peers
due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist
in the remote cluster
> 2017-07-31 06:25:25,798 INFO [Clustering Tasks Thread-1] o.a.n.c.c.ClusterProtocolHeartbeater
Heartbeat created at 2017-07-31 06:25:25,790 and sent to nifi1:10600 at 2017-07-31 06:25:25,798;
send took 6 millis
> 2017-07-31 06:25:27,962 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
Could not communicate with nifi1:8081 to determine which nodes exist in the remote NiFi cluster,
due to org.apache.http.conn.HttpHostConnectException: Connect to nifi1:8081 [nifi1/10.0.0.32]
failed: Connection refused (Connection refused)
> 2017-07-31 06:25:27,962 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
org.apache.nifi.remote.client.PeerSelector@4516d212 Unable to refresh Remote Group's peers
due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist
in the remote cluster
> {code}
> This is unexpected as the node address is wrong -- it should be nifi1:8080 instead of
nifi1:8082 (being the port of the local machine I have taken the warning from). The same error
I can see in the logs of nifi2 (for peer "nifi1:8081").



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message