Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Fri, 4 Aug 2017 21:13:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@nifi.apache.org
Message-ID: <JIRA.13084901.1499275753000.97413.1501881180141@Atlassian.JIRA>
In-Reply-To: <JIRA.13084901.1499275753000@Atlassian.JIRA>
References: <JIRA.13084901.1499275753000@Atlassian.JIRA> <JIRA.13084901.1499275753262@jira-lw-us.apache.org>
Subject: [jira] [Commented] (NIFI-4152) Create ListenTCPRecord Processor
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 04 Aug 2017 21:13:05 -0000


    [ https://issues.apache.org/jira/browse/NIFI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114973#comment-16114973 ] 

ASF GitHub Bot commented on NIFI-4152:
--------------------------------------

Github user pvillard31 commented on the issue:

    https://github.com/apache/nifi/pull/1987
  
    Hey @bbende, just built this PR and did some tests. 
    
    The template I used is here:
    https://gist.github.com/pvillard31/5ecea5932bf70fc622e30be8512601b6
    
    Then I send messages using nc:
    ````
    $> nc localhost 9876
    2016-11-08 21:24:23,029 FINE Yellow
    2016-11-08 21:24:23,029 INFO Test Message 1
    2016-11-08 21:24:23,029 WARN Red
    2016-11-08 21:24:23,029 ERROR Green
    2016-11-08 21:24:23,029 FATAL Blue
    ...
    ````
    
    Observations:
    - If I start ``nc`` but don't send any message, it'll fail after the read timeout with the following message (and kill my ``nc`` connection):
    ````
    ListenTCPRecord[id=aefaaba3-015d-1000-cecb-dd76899c8193] Error processing records: null: java.net.SocketTimeoutException
    ````
    I'm not sure if we want to raise a bulletin alert if the source is not sending any message. Thoughts?
    
    - If I start ``nc`` and send messages, but less than the maximum number of records per flow file (1000), then it'll generate a flow file only after 30 seconds without receiving any message (and it'll generate a bulletin).
    
    - If I start ``nc`` and keep sending messages, it'll generate flow files containing exactly the number set for the property "record batch size".
    
    While I understand this is the intended behaviour, I'm not sure if this is clear enough in the description of the processor. In particular, raising a bulletin and killing the connection in case no data is received could seem weird (and it's not similar to the way ListenTCP is working).
    
    Instead of killing the connection, what about just generating a flow file with the amount data available at this moment? Or change the level of the log message to info? And probably improve the message to let the user knows that the connection has been closed because no data has been received since X seconds? Thoughts?


> Create ListenTCPRecord Processor
> --------------------------------
>
>                 Key: NIFI-4152
>                 URL: https://issues.apache.org/jira/browse/NIFI-4152
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Bryan Bende
>            Assignee: Bryan Bende
>            Priority: Minor
>         Attachments: ListenTCPRecordWithGrok.xml
>
>
> We should implement a ListenTCPRecord that can pass the underlying InputStream from a TCP connection to a record reader.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)