flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <till.rohrm...@gmail.com>
Subject Re: Figuring out when a job has successfully restored state
Date Wed, 29 Mar 2017 08:22:58 GMT
Hi Gyula,

there exists a related issue [1]. Fixing this issue will move the state
restoration in the state DEPLOYING. This means that when you see a task
being in state RUNNING, then it will have restored all of its eager state.

[1] https://issues.apache.org/jira/browse/FLINK-4714


On Tue, Mar 28, 2017 at 10:55 AM, Gyula Fóra <gyula.fora@gmail.com> wrote:

> Hi,
> Another thought I had last night, maybe we could have another state for
> recovering jobs in the future.
> Deploying -> Recovering -> Running
> This recovering state might only be applicable for state backends that
> have to be restored before processing can start, lazy state backends (like
> external databases) might go into processing state "directly".
> What do you think? (I'm ccing dev)
> Gyula
> Gyula Fóra <gyula.fora@gmail.com> ezt írta (időpont: 2017. márc. 27., H,
> 17:06):
>> Hi all,
>> I am trying to figure out the best way to tell when a job has
>> successfully restored all state and started process.
>> My first idea was to check the rest api and the number of processed bytes
>> for each parallel operator and if thats greater than 0, it started.
>> Unfortunately this logic fails if the operator doesnt receive any input for
>> some time.
>> Do we have any info like this exposed somewhere in a nicely queryable way?
>> Thanks,
>> Gyula

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message