flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Question about SlidingPreReducers
Date Wed, 29 Apr 2015 17:09:22 GMT
Oh, I saw them, but in updateCurrent() (both grouped and non-grouped)
the else path is never taken when I run the windowed WordCount
example.

I have a branch where I just throw a RuntimeException in the else
branches and the whole things still works: mvn verify doesn't fail,
the examples still run. When looking at the behaviour of the
functions, it seems clear to me that the else path can never be taken.

On Wed, Apr 29, 2015 at 6:15 PM, Gyula Fóra <gyfora@apache.org> wrote:
> Hey,
>
> They actually work :P Although I have to admit I need to do some
> refactoring of the method names and parameters.
>
> I made some quick refactoring and added some comments for the key methods:
>
> https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L96
>
> https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L105
>
> https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L132
>
> Those three parts contain the key logic, and now the method names should
> make more sense as well :)
>
> Cheers,
> Gyula
>
>
>
>
>
> On Wed, Apr 29, 2015 at 4:44 PM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
>
>> Hi,
>> one of my notorious (by now) investigative questions: How do these
>> work? To meet it seems that they do not perform any actual pre
>> aggregation but just keep all the elements in a LinkedList
>> (non-grouped case) or HashMap (grouped-case) and aggregate the
>> elements when the window is to be emitted.
>>
>> Am I missing something again? :D
>>
>> Cheers,
>> Aljoscha
>>

Mime
View raw message