flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Interpretation of Trigger and Eviction on a window
Date Mon, 30 Nov 2015 14:29:20 GMT
Hi,
the Evictor is very tricky to understand, I’m afraid. What happens when a Trigger fires
is the following:
 1. Trigger fires
 2. Evictor can remove elements from the window buffer
 3. Window function processes the elements that remain in the window buffer

The tricky thing here is that the Evictor should really be called “Keeper”. What it does
in fact is specify how many elements should be kept in the buffer. For example, an Evictor
“CountEvict(5)” means, keep 5 elements, a “TimeEvictor(5 minutes)” keeps the elements
that a younger than 5 minutes. I admit this is somewhat counterintuitive but this is how the
policies work in IBM Infosphere Streams and they influenced the early design of the Flink
Trigger/Eviction policies. (See here for a description of the semantics of IBM Infosphere
Streams: http://www.cs.bilkent.edu.tr/~bgedik/homepage/lib/exe/fetch.php/wiki:pubs:windowing.pdf)

If it is still unclear, could you please but together a working example where the behavior
is unclear to you so that we can have a look at it.

Cheers,
Aljoscha
> On 30 Nov 2015, at 12:08, Nirmalya Sengupta <sengupta.nirmalya@gmail.com> wrote:
> 
> Hi Aljoscha <aljoscha@apache.org>,
> 
> Thanks for taking interest in my post and question.
> 
>  If the Evictor removes elements  _before_ the function is applied, then what happens
the first time, the Evictor is acting? That's what I am failing to understand. At the beginning
of the operation on the Stream, he Trigger finds 5 elements, the Evictor removes 2 of them
(let's say), and Sum sees incorrect number of elements. Isn't this possible or again, am I
demonstrating my loose grasp of the matter? 
> 
> Please correct me.
> 
> -- N
> 
> 
> -- 
> Software Technologist
> http://www.linkedin.com/in/nirmalyasengupta
> "If you have built castles in the air, your work need not be lost. That is where they
should be.
> Now put the foundation under them."


Mime
View raw message