beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <>
Subject [jira] [Commented] (BEAM-1674) Flink user state GC depends on order of timer firing
Date Thu, 09 Mar 2017 15:18:38 GMT


Aljoscha Krettek commented on BEAM-1674:

[~lzljs3620320] Do you maybe have an idea about this?

I think we can add a complicated scheme of piggybacking several timers on one "physical" Flink
timer but we would need to manage those and keep some state to determine what timer ids we
have to fire for when a physical Flink timer fires.

> Flink user state GC depends on order of timer firing
> ----------------------------------------------------
>                 Key: BEAM-1674
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 0.5.0
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 0.6.0
> The newly added {{ParDoTest.testEventTimeTimerMultipleKeys()}} fails because the {{DoFn}}
sets a timer for {{window.maxTimestamp()}} which also happens to be the GC timer for the user
state. The Flink Runner uses timers to schedule GC, the user-set timer and the GC timer have
a different timer id, so they don't clash. However, if the GC timer is being processed before
the user timer then the user doesn't have a chance to access the state anymore because it
will already be cleared out by the time the user timer is being processed. 

This message was sent by Atlassian JIRA

View raw message