nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davy De Waele <ddewa...@gmail.com>
Subject Re: PutTCP connector not cleaning up dangling connections
Date Wed, 20 Sep 2017 19:45:34 GMT
Don't get it either... new code looks more or less the same. been trying
various things all day. Glad I got it working with a "patched" PutTCP.
Better from what I got but still not ideal.

Can it be a threading issue (we interact with multiple modules) in the
input / output stream handling ? Also noticed a flush that wasn't there
before. Are there other places where charsets can be set ?

Been thinking about what could cause the tcp server to react this way but
no idea. Can reproduce it easily. Will check if I can enable additional
logging on the moxa.

On Wed, 20 Sep 2017 at 21:36, Bryan Bende <bbende@gmail.com> wrote:

> For what it is worth, I can't reproduce this on my Mac laptop either...
>
> Setup GenerateFlowFile with same sample text your provided, followed
> by PutTCP with the \r\n delimiter, this sends to ListenTCP in the same
> NiFi instance.
>
> If I look at the content of the flow files coming out of ListenTCP,
> and I put the content viewer in Hex mode, I see:
>
> 0x00000000 0D 40 43 4F 4D 4D 41 4E 44 0D 0D .@COMMAND..
>
> The 0D 40 43 4F 4D 4D 41 4E 44 0D 0D lines up with \r@COMMAND\r  which
> would make sense since ListenTCP read-off the \r\n at the end.
>
>
> On Wed, Sep 20, 2017 at 1:20 PM, Bryan Bende <bbende@gmail.com> wrote:
> > I'll keep looking at this, but just to summarize the changes that were
> > made in case something jumps out to someone else...
> >
> > The changes were to avoid reading the entire content of the flow file
> > into memory.
> >
> > Previous Behavior:
> > - Create a ByteArrayOuputStream
> > - Use an InputStreamCallback to read entire bytes of flow file, and
> > write them to above output stream
> > - Get value of the outgoing delimiter property
> > - If delimiter is not null, then get the bytes of the delimiter using
> > the charset configured in the processor, and write the delimiter bytes
> > to the above output stream
> > - Get byte array from the ByteArrayOutputStream and pass it to the
> > channel sender, which in turns calls write on a SocketChannel
> >
> > New behavior:
> > - Get the direct SockeOutputStream from the sender
> > - Get the InputStream of the flow file
> > - Call IOUtils.copy to copy the flow file input stream to the socket
> > output stream in chunks
> > - Perform exact same delimiter logic as above
> >
> >
> >
> >
> > On Wed, Sep 20, 2017 at 1:13 PM, Davy De Waele <ddewaele@gmail.com>
> wrote:
> >> Correct about the delimiter.
> >> Think character set is (was) automatically filled in with utf-8. (Never
> >> noticed a default option)
> >>
> >> On Wed, 20 Sep 2017 at 19:11, Bryan Bende <bbende@gmail.com> wrote:
> >>>
> >>> Thanks for the info and for digging into this..
> >>>
> >>> Just to confirm, in your PutTCP configuration, you are saying that you
> >>> have "Outgoing Message Delimiter" set to \r\n  ?
> >>>
> >>> And do you have the "Character Set" property set to the default of
> "UTF-8"
> >>> ?
> >>>
> >>> On Wed, Sep 20, 2017 at 12:59 PM, ddewaele <ddewaele@gmail.com> wrote:
> >>> > I've put up a git repo that I use to test the PutTCP behavior.
> >>> >
> >>> >
> >>> > (
> https://github.com/ddewaele/com.ixortalk.tcp/compare/dev_puttcp_1.4?expand=1
> )
> >>> >
> >>> > What I've done.
> >>> >
> >>> > - Built Nifi 1.4.0-SNAPSHOT + your PR and used that as my base nifi
> >>> > installation
> >>> > - Noticed weird characters in the tcp responses
> >>> > - Switched to 1.3.0. No more weird characters
> >>> > - Isolated the PutTCP processor by creating a custom PutTCP (see
> github
> >>> > repo
> >>> > below) to experiment (instead of the built-in PutTCP)
> >>> > - Repo has 2 branches. master contains an "older" PutTCP processor
> from
> >>> > the
> >>> > Nifi code base. With that one I don't have the weird characters
> (even on
> >>> > a
> >>> > nifi 1.4.0-SNAPSHOT + your PR).
> >>> > - If I use the dev_puttcp_1.4 branch on my nifi I get the weird
> >>> > characters.
> >>> >
> >>> >
> >>> > Some info on my setup :
> >>> >
> >>> > I'm sending commands to PutTCP using a GenerateFlowFile processor.
> >>> > payload = ${literal('\r')}@COMMAND${literal('\r')}
> >>> > PutTCP adds a delimier (\r\n)
> >>> >
> >>> > Regarding provenance. PutTCP adds the delimiter to the payload before
> >>> > sending it to the TCP socket. That  delimiter is not in the flowfile
> >>> > that is
> >>> > put in provenance I think.
> >>> >
> >>> > Cannot reproduce the issue on my mac with my mock tcp server. But on
> the
> >>> > actual environment (Linux CentOS) with the Moxa devices, there is a
> >>> > clear
> >>> > difference in behaviour. Will see if I can tunnel my through the
> devices
> >>> >
> >>> > All the info I have right now ....
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
>

Mime
View raw message