flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nirmalya Sengupta <sengupta.nirma...@gmail.com>
Subject Re: Finding the average temperature
Date Tue, 16 Feb 2016 01:19:29 GMT
Hello Stefano <stefano.baghino@radicalbit.io>

Sorry for the late reply. Many thanks for taking effort to write and share
an example code snippet.

I have been playing with the countWindow behaviour for some weeks now and I
am generally aware of the functionality of countWindowAll(). For my
useCase, where I _have to observe_ the entire stream as it founts in, using
countWindowAll() is probably the most obvious solution. This is what you
recommend too. However, because this is going to use 1 thread only (or 1
node only in a cluster), I was thinking about ways to make use of the
'distributedness' of the framework. Hence, my question.

Your reply leads to me read and think a bit more. If I have to use
parallelism to achieve what I want to achieve, I think managing a
ValueState of my own is possibly the solution. If you have any other
thoughts, please share.

>From your  earlier response: '... you can still enjoy a high level of
parallelism up until the last operator by using a combiner, which is
basically a reducer that operates locally ...'. Could you elaborate this a
bit, whenever you have time?

-- Nirmalya

Software Technologist
"If you have built castles in the air, your work need not be lost. That is
where they should be.
Now put the foundation under them."

View raw message