flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aitozi <...@git.apache.org>
Subject [GitHub] flink issue #5405: [FLINK-8477][Window]Add api to support user to skip serva...
Date Sun, 11 Feb 2018 08:29:13 GMT
Github user Aitozi commented on the issue:

    Hi @aljoscha , you have mentioned two points : 
    1. The events arrived may out of order in event-time processing 
    2. We can use windowFunction or ProcessWindowFunction to filter serverl window by specify
the start time of window and the endtime.
    I have some differerent ideas: 
    1. when we deal with the out-of-order eventtime stream, we may specify the maxOutOfOrder
to avoid the too much late elements skipped, so when the job restart/start the maxNumOfWindow
to be skipped can be set to  maxOutOfOrder/(the length of the thumbling window), So that the
late elements will not produce incorrect results. The num of the window need to be skipped
is according to the degree of the out of order
    2. We need to skip the serveral broken window data , and we dont know which window is
broken, we can just detect which window is first fired and the serval window after this is
broken too. The num should very from the production (according to the maxOutOfOrder &
the length of the window )


View raw message