flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-7229) Flink doesn't deleted old checkpoint
Date Wed, 19 Jul 2017 13:15:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Zhou updated FLINK-7229:
------------------------------
    Description: 
I have a 6-node Flink cluster where one contains jobmanager and the others have only taskmanagers.
All taskmanagers have the following config :

state.backend: rocksdb
state.backend.rocksdb.checkpointdir: file:///opt/flink/data/local-checkpoints
state.backend.fs.checkpointdir: file:///opt/flink/data/local-checkpoints
state.checkpoints.dir: file:///opt/flink/data/glusterfs/external-checkpoints
state.checkpoints.num-retained: 3

And both checkpoints and external-checkpoints are enabled in the application.  

After running the application for a while, I can see that Flink do retains metadata for 3
checkpoints in external-checkpoints. However, for the local-checkpoint, only one node(with
jobmanager) retains 3 checkpoints and the others don't delete old checkpoints. This causes
the issue when the other nodes are junked with checkpoints and I have to manually clean them
up.


  was:
I have a 6-node Flink cluster where one contains jobmanager and the others have only taskmanagers.
All taskmanagers have the following config :

state.backend: rocksdb
state.backend.rocksdb.checkpointdir: file:///opt/flink/data/local-checkpoints
state.backend.fs.checkpointdir: file:///opt/flink/data/local-checkpoints
state.checkpoints.dir: file:///opt/flink/data/glusterfs/external-checkpoints
state.checkpoints.num-retained: 3

And both checkpoints and external-checkpoints are enabled in the application.  

I can see that for the external-checkpoint, Flink do retains metadata for 3 checkpoints. However,
for the local-checkpoint, only one node(with jobmanager) retains 3 checkpoints and the others
don't delete old checkpoints. This causes the issue when the other Flink nodes are junked
with checkpoints and I have to manually clean them up.



> Flink doesn't deleted old checkpoint
> ------------------------------------
>
>                 Key: FLINK-7229
>                 URL: https://issues.apache.org/jira/browse/FLINK-7229
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.3.1
>         Environment: Six Flink nodes running on Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-121-generic
x86_64)
>            Reporter: Jason Zhou
>
> I have a 6-node Flink cluster where one contains jobmanager and the others have only
taskmanagers. All taskmanagers have the following config :
> state.backend: rocksdb
> state.backend.rocksdb.checkpointdir: file:///opt/flink/data/local-checkpoints
> state.backend.fs.checkpointdir: file:///opt/flink/data/local-checkpoints
> state.checkpoints.dir: file:///opt/flink/data/glusterfs/external-checkpoints
> state.checkpoints.num-retained: 3
> And both checkpoints and external-checkpoints are enabled in the application.  
> After running the application for a while, I can see that Flink do retains metadata for
3 checkpoints in external-checkpoints. However, for the local-checkpoint, only one node(with
jobmanager) retains 3 checkpoints and the others don't delete old checkpoints. This causes
the issue when the other nodes are junked with checkpoints and I have to manually clean them
up.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message