flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Puneet Kinra <puneet.ki...@customercentria.com>
Subject Re: Forking a stream with Flink
Date Tue, 29 Jan 2019 15:24:24 GMT
Hi Selvaraj

In your pojo add data member as status or something like that,now set it
error in case it is invaild .pass the output of flatmap
to split opertor there you can split the stream

On Tue, Jan 29, 2019 at 6:39 PM Selvaraj chennappan <
selvarajchennappan@gmail.com> wrote:

> UseCase:- We have kafka consumer to read messages(json ) then it applies
> to flatmap  for transformation based on the rules ( rules are complex ) and
> convert it to pojo .
> We want to verify the record(pojo) is valid by checking field by field of
> that record .if record is invalid due to transformation rules  then move to
> error topic otherwise send to DB.
>
> I thought of Implementing like adding another consumer to read json
> message  and compare json message attributes with transformed record
> attributes .
>
> Hence I need to join/coprocess these two streams to validate then decide
> whether persist to db or sending to error topic.
>
> Please let me know if you need more information.
>
> On Tue, Jan 29, 2019 at 6:21 PM miki haiat <miko5054@gmail.com> wrote:
>
>> Im not sure if i got your question correctly, can you elaborate more on
>> your use case
>>
>
>
> --
>
>
>
>
>
> Regards,
> Selvaraj C
>


-- 
*Cheers *

*Puneet Kinra*

*Mobile:+918800167808 | Skype : puneet.kinra@customercentria.com
<puneet.kinra@customercentria.com>*

*e-mail :puneet.kinra@customercentria.com
<puneet.kinra@customercentria.com>*

Mime
View raw message