flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Splitting a stream based on validation
Date Tue, 21 Feb 2017 12:04:21 GMT
I think you basically need something like this:

DataStream input = ...
DataStream withErrors = input.filter(new MyErrorFilter());
DataStream withoutErrors = input.filter(new MyWithoutErrorFilter());

withErrors.addSink(...)
withoutErrors.addSink(...)

Does that help?

On Mon, 20 Feb 2017 at 13:44 Chet Masterson <chet.masterson@yandex.com>
wrote:

>
> A while back on the mailing list, there was a discussion on validating a
> stream, and splitting the stream into two sinks, depending on how the
> validation went:
>
> (operator generating errors)
>     --> (filter) --> stream without errors --> sink
>     --> (filter) --> error stream  --> sink
>
> Is there an example of this implemented in (scala) code anywhere? I'm not
> sure how to code this up. Do I embed the error sink in the filter? The
> compiler hated everything I tried.
>

Mime
View raw message