flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Flink CEP with event time
Date Wed, 03 Jan 2018 14:36:12 GMT
Ok, but will there be events in all Kafka partitions/topics?

> On 3. Jan 2018, at 15:33, shashank agarwal <shashank734@gmail.com> wrote:
> 
> Hi,
> 
> Yes, Events will always carry a variable OriginTimestamp which I am using in the extractor.
I have used fallback also in case of data missing will put System current millis. 
> 
> Still, it's not printing results.
> 
> Best,
> Shashank
> 
> 
> 
> 
> 
> ‌
> 
> On Wed, Jan 3, 2018 at 7:40 PM, Aljoscha Krettek <aljoscha@apache.org <mailto:aljoscha@apache.org>>
wrote:
> Hi,
> 
> Are all the partitions always carrying data that has advancing timestamps? When using
Event-time the Kafka source (and Flink in general) needs to have steady progress in all partitions,
otherwise the watermark does not advance, which in turn means that processing will be stalled
downstream.
> 
> Best,
> Aljoscha
> 
> 
>> On 3. Jan 2018, at 14:29, shashank agarwal <shashank734@gmail.com <mailto:shashank734@gmail.com>>
wrote:
>> 
>> Hello,
>> 
>> I have some patterns in my program. For an example,
>> 
>>  A followedBy B.
>> 
>> As I am using kafka source and my event API's using load balancers so sometimes B
comes before A. So my CEP doesn't generate any result for those events. 
>> 
>> I have then tried event time and applied "BoundedOutOfOrdernessTimestampExtractor"
on kafkasource with extract time from an origin time variable which I have in the event. I
am using watermark lateness of 10 seconds in that.
>> 
>> Now CEP stopped generating results. It's not even generating results where Event
B comes after A. I have tried within (10 seconds) in CEP also still not generating results.

>> 
>> Am I doing anything wrong?
>> 
>> I have to cover the case where B can come after A from Kafka.
>> 
>> -- 
>> Thanks Regards
>> 
>> SHASHANK AGARWAL
>>  ---  Trying to mobilize the things....
>> 
>> 
>> 
>> 
>> ‌
> 
> 
> 
> 
> -- 
> Thanks Regards
> 
> SHASHANK AGARWAL
>  ---  Trying to mobilize the things....


Mime
View raw message