flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2929) Recovery of jobs on cluster restarts
Date Mon, 02 Nov 2015 00:12:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984605#comment-14984605

Till Rohrmann commented on FLINK-2929:

I'm wondering how often it's the case that someone, who stops the cluster and then starts
it again (after some time), expects that his old jobs are still there. This might be the case
when you upgrade your cluster. But when you run your job and stop your cluster, then I would
expect that everything has been stopped.

> Recovery of jobs on cluster restarts
> ------------------------------------
>                 Key: FLINK-2929
>                 URL: https://issues.apache.org/jira/browse/FLINK-2929
>             Project: Flink
>          Issue Type: Improvement
>    Affects Versions: 0.10
>            Reporter: Ufuk Celebi
> Recovery information is stored in ZooKeeper under a static root like {{/flink}}. In case
of a cluster restart without canceling running jobs old jobs will be recovered from ZooKeeper.
> This can be confusing or helpful depending on the use case.
> I suspect that the confusing case will be more common.
> We can change the default cluster start up (e.g. new YARN session or new ./start-cluster
call) to purge all existing data in ZooKeeper and add a flag to not do this if needed.
> [~trohrmann@apache.org], [~aljoscha], [~StephanEwen] what's your opinion?

This message was sent by Atlassian JIRA

View raw message