flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Calculating metrics real time
Date Thu, 05 Oct 2017 12:43:54 GMT
That depends on the state backend [1] that you are using.
If you use the RocksDBStateBackend, state is written to RocksDB which
persists to disk.

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/ops/state_backends.html

2017-10-05 14:41 GMT+02:00 Rahul Raj <rahulrajmsrit@gmail.com>:

> Thanks for the response. So, i am guessing windows in flink will store the
> records in memory before processing them. Correct?
>
> Rahul Raj
>
> On Oct 5, 2017 17:50, "Fabian Hueske" <fhueske@gmail.com> wrote:
>
>> Hi,
>>
>> I'd suggest to have a look at the window operators [1]. For example a
>> tumbling window of 1 minute can be used to compute metrics every minute.
>> Flink's window operators are very extensible and you can implement very
>> custom logic if the predefined windows don't match your use case. In any
>> case, windows are used to collect and perform a computation on a set of
>> records.
>>
>> Best, Fabian
>>
>> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/
>> dev/windows.html
>>
>> 2017-10-05 12:57 GMT+02:00 Rahul Raj <rahulrajmsrit@gmail.com>:
>>
>>> Hi,
>>>
>>> I have to calculate some complicated metrics like click through rate ,
>>> click value rate and conversions on real time data using flink. But I am
>>> not sure what functionality of flink should I use to program this because
>>> it involves collection of some records in memory for certain time may be 1
>>> minute and then using formula for calculating metrics on those records.
>>>
>>> Am I correct with my approach or is there any preferred approach for
>>> such tasks in flink? Can I use windows for doing this? Any tutorial or
>>> example would be really great.
>>>
>>> Rahul Raj
>>>
>>
>>

Mime
View raw message