nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: conditional Listen Syslog
Date Wed, 06 Jul 2016 20:32:34 GMT
Hello

Is this syslog via UDP or is it via TCP?  If it is UDP then there
isn't much we can do to push back on the sender.  If it is TCP then we
certainly will stop bringing in new data which in turn should
effectively push back but I'm not sure that will work well because the
sender may not be able to buffer long.  Another consideration is that
NiFi itself can be used as a rather large buffer for this data itself.

Alternatively, you might find that this is a good use case for Kafka
as a buffer where it is syslogsender -> nifi -> kafka and then the
spark jobs source data from kafka.  That won't offer back pressure but
achieving true back pressure in syslog based sources broadly might be
elusive anyway.

Thanks
Joe


On Wed, Jul 6, 2016 at 12:14 PM, pradeepbill <pradeep.bill@gmail.com> wrote:
> Thanks David, I have done that, and I still see a connection being
> established between the ListenSysLog port and the server  that is pushing
> data to ListenSysLog, then I thought NIFI is ignoring/loosing data after
> enabling back pressure, am I wrong?, I would like to be wrong ? Please
> advice.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/conditional-Listen-Syslog-tp12652p12655.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message