nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raf Huys (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (NIFI-2615) Add support for GetTCP processor
Date Wed, 28 Sep 2016 14:12:20 GMT

    [ https://issues.apache.org/jira/browse/NIFI-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529724#comment-15529724
] 

Raf Huys edited comment on NIFI-2615 at 9/28/16 2:11 PM:
---------------------------------------------------------

Hi [~apsaltis], we're having the same requirement: we need a TCP connection where we can write
to *and* read from. Transformed to NiFi, that would mean a PutTCP and a GetTCP.  

I found your release of the nifi-standard-nar (https://github.com/apsaltis/nifi-gettcp/releases).
I'm using nifi@1.0.0 now, and replacing the nifi-standard-nar-1.0.0 with your 0.6.1 gives

{code}
2016-09-28 15:51:18,626 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.standard.GetTCP could
not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.standard.GetTCP
could not be instantiated
	at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_101]
	at ...
Caused by: java.lang.NoSuchMethodError: org.apache.nifi.processors.standard.GetTCP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;
	at org.apache.nifi.processors.standard.GetTCP.<init>(GetTCP.java:133) ~[nifi-standard-processors-0.6.1.jar:0.6.1]
{code}

- do you plan to push this GetTCP feature upstream? If not, any reason why?
- are there plan to open source the release versions of GetTCP?


was (Author: raf.huys@gmail.com):
Hi [~apsaltis], we're having the same requirement: we need a TCP connection where we can write
to *and* read from. Transformed to NiFi, that would mean a PutTCP and a GetTCP.  

I found your release of the nifi-standard-nar (https://github.com/apsaltis/nifi-gettcp/releases).
I'm using nifi@1.0.0 now, and replacing the nifi-standard-nar-1.0.0 with your 0.6.1 gives

{code}
2016-09-28 15:51:18,626 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.standard.GetTCP could
not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.standard.GetTCP
could not be instantiated
	at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_101]
	at ...
Caused by: java.lang.NoSuchMethodError: org.apache.nifi.processors.standard.GetTCP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;
	at org.apache.nifi.processors.standard.GetTCP.<init>(GetTCP.java:133) ~[nifi-standard-processors-0.6.1.jar:0.6.1]
{code}

- do you plan to push this GetTCP feature upstream? If not, any reason why?

> Add support for GetTCP processor
> --------------------------------
>
>                 Key: NIFI-2615
>                 URL: https://issues.apache.org/jira/browse/NIFI-2615
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Core Framework
>    Affects Versions: 1.0.0, 0.7.0, 0.6.1
>            Reporter: Andrew Psaltis
>            Assignee: Andrew Psaltis
>
> This processor will allow NiFi to connect to a host via TCP, thus acting as the client
and consume data. This should provide the following properties:
> * Endpoint -  this should accept a list of addresses in the format of <Address>:<Port>
- if a user wants to be able to track the ingestion rate per address then you would want to
have one address in this list. However, there are times when multiple endpoints represent
a logical entity and the aggregate ingestion rate is representative of it. 
> * Failover Endpoint - An endpoint to fall over to if the list of Endpoints is exhausted
and a connection cannot be made to them or it is disconnected and cannot reconnect.
> * Receive Buffer Size -- The size of the TCP receive buffer to use. This does not related
to the size of content in the resulting flow file.
> * Keep Alive -- This enables TCP keep Alive
> * Connection Timeout -- How long to wait when trying to establish a connection
> * Batch Size - The number of messages to put into a Flow File. This will be the max number
of messages, as there may be cases where the number of messages received over the wire waiting
to be emitted as FF content may be less then the desired batch.
> This processor should also support the following:
> 1. If a connection to end endpoint is broken, it should be logged and reconnections to
it should be made. Potentially an exponential backoff strategy will be used. The strategy
if there is more than one should be documented and potentially exposed as an Attribute.
> 2. When there are multiple instances of this processor in a flow and NiFi is setup in
a cluster, this processor needs to ensure that received messages are not dual processed. For
example if this processor is configured to point to the endpoint (172.31.32.212:10000) and
the data flow is running on more than one node then only one node should be processing data.
In essence they should form a group and have similar semantics as a Kafka consumer group does.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message