flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shashank agarwal <shashank...@gmail.com>
Subject Re: Process event with last 1 hour, 1week and 1 Month data
Date Thu, 15 Jun 2017 15:47:09 GMT
Thanks Aljoscha Krettek I will try the same.

On Thu, Jun 15, 2017 at 3:11 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi,
>
> How would you evaluate such a query? I think the answer could be that you
> have to keep all that older data around so that you can evaluate when a new
> event arrives. In Flink, you could use a ProcessFunction for that and use a
> MapState that keeps events bucketed into one-week intervals. This way, can
> more efficiently iterate over the buckets that are required when evaluating
> a given event and you can also efficiently delete a complete bucket of
> older events once you know that they are not required anymore.
>
> These are the relevant sections of the Flink doc:
>  - https://ci.apache.org/projects/flink/flink-docs-
> release-1.3/dev/stream/state.html
>  - https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/
> process_function.html
>
> Best,
> Aljoscha
>
> On 13. Jun 2017, at 15:27, shashank agarwal <shashank734@gmail.com> wrote:
>
> Hi,
>
> I have to process each event with last 1 hour , 1 week and 1 month data.
> Like how many times same ip occurred in last 1 month corresponding to that
> event. \
>
> I think window is for fixed time i can't calculate with last 1 hour
> corresponding to current event.
>
> If you have any clue please guide what should i use Table, ProcessFunction
> or global window. Or what approach should i take ?
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
>  ---  Trying to mobilize the things....
>
>
>


-- 
Thanks Regards

SHASHANK AGARWAL
 ---  Trying to mobilize the things....

Mime
View raw message