flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nirmalya Sengupta <sengupta.nirma...@gmail.com>
Subject Interpretation of Trigger and Eviction on a window
Date Fri, 27 Nov 2015 12:25:45 GMT
Hello there.

I have just started exploring Apache Flink, and it has immediately got me
excited. Because I am a beginner, my questions may be a bit naive. Please
bear with me.

I refer to this particular sentence from Flink 0.10.0 Guide
<https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html#windows-on-unkeyed-data-streams>
:

' *After the trigger fires, and before the function (e.g., sum, count) is
applied to the window contents, an optional Evictor removes some elements
from the beginning of the window before the remaining elements are passed
on to the function* '

I am a bit confused with the assertion that elements are evicted *before*
the function is applied. Let me elaborate what my understanding is.

Let us say that my window has a 'count' trigger of 10 elements, with some
time-pane of 2 seconds (assumption: rate of ingestion is fast enough for at
least 10 elements to arrive within 2 seconds).

windowedStream.*trigger*(CountTrigger.of(10)).*evictor*(CounEvictor.of(10)).sum(_._1)
// summation of the 2nd field of a tuple

Now, till the time 9 elements have gathered in the window, the trigger is
dormant. After the 10th element  enters the window-pane, the trigger is
fired. At this point in time, all these 10 elements should be passed to the
_sum_ function so that correct summated value is generated and **only
then** the evictor is allowed to take out all the 10 elements leaving the
window-pane empty. The window's element count is set to zero and  it awaits
the arrival of the next element.

However, what the documents seems to suggest is that the evictor will be
able to take out _some_ (how many?) elements from the _beginning_ of the
window, before the _sum_ function can see the elements. Isn't this
counterintuitive or am I missing something obvious here?

Will keenly wait to hear from you.

-- Nirmalya

-- 
Software Technologist
http://www.linkedin.com/in/nirmalyasengupta
"If you have built castles in the air, your work need not be lost. That is
where they should be.
Now put the foundation under them."

Mime
View raw message