flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yun Tang <myas...@live.com>
Subject Re: Flink 1.6, increment Checkpoint, the shared dir stored the last year checkpoint state
Date Sun, 19 Jan 2020 19:33:19 GMT
Hi Lake

A more suitable place for this mail should be in user-mail list.

There are three reasons why this could happen:

  1.  This file is orphan file e.g. file uploaded during one checkpoint but task manager exited
unexpectedly leave that checkpoint not completed.
  2.  This file should be removed by checkpoint coordinator but takes too long to complete
before job shut down.
  3.  This file is still useful. This is possible in theory because some specific rocksDB
sst file might not be selected during compactions for a long time.

Best
Yun Tang
________________________________
From: LakeShen <shenleifighting@gmail.com>
Sent: Sunday, January 19, 2020 18:55
To: user <user@flink.apache.org>; user-zh@flink.apache.org <user-zh@flink.apache.org>;
dev <dev@flink.apache.org>
Subject: Flink 1.6, increment Checkpoint, the shared dir stored the last year checkpoint state

Hi community´╝î
        now I have a flink sql job, and I set the flink sql sate retention time, there are
three dir in flink checkpoint dir :
1. chk -xx dir
2. shared dir
3. taskowned dir

I find the shared dir store the last year checkpoint state,the only reason I thought is that
the latest
checkpoint retain reference of last year checkpoint state file.
Are there any other reason to lead this? Or is it a bug?

Thanks to your replay.

Best wishes,
Lake Shen


Mime
View raw message