flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 魏偉哲 <tony19920...@gmail.com>
Subject Re: Question on window ProcessFunction
Date Wed, 26 Apr 2017 12:51:06 GMT
Hi Aljoscha,

I know the aggregate code is newer. I am confused because the
implementations are not consistent.
Does it mean that the reduce/fold implementation would need to be
refactored for the purpose of having less layers ?
Or is it better to remain the current implementations for some reasons?

Many thanks,
Tony Wei

2017-04-26 20:24 GMT+08:00 Aljoscha Krettek <aljoscha@apache.org>:

> Hi Tony,
> The reason for this is that the aggregate code is newer. The new code has
> less layers, compared to the reduce/fold implementation where it is
> InternalFunction(ReduceApplyFunction(Reduce)) instead of
> InteralAggregateFunction(Aggregate).
>
> Best,
> Aljoscha
> > On 26. Apr 2017, at 06:39, 魏偉哲 <tony19920430@gmail.com> wrote:
> >
> > Hi all,
> >
> > Recently, I was tracing the source code in streaming api and I was
> confused
> > about some implementations.
> >
> > When using reduce function with evictor, the *WindowStream* will wrap the
> > *ReduceFunction* and *ProcessWindowFunction* into
> > *ReduceApplyProcessWindonwFunction* and put it in
> > *InternalIterableProcessWindowFunction*. So does fold function.
> >
> > However, when using aggregate, the *InternalIterableProcessWindowF
> unction*
> > was changed to *InternalAggregateProcessWindowFunction* which was
> applied
> > aggregation in the process() method.
> >
> > My question is why not implement an *AggregateApplyProcessWindowFun
> ction*
> > and use *InternalIterableProcessWindowFunction* instead just like
> reduce,
> > fold function did. Is there any concern?
> >
> > Many thanks,
> > Tony Wei
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message