flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yun Tang <myas...@live.com>
Subject Re: Increased Size of Incremental Checkpoint
Date Thu, 06 Sep 2018 06:54:01 GMT
+ user mail list
________________________________
From: Yun Tang <myasuka@live.com>
Sent: Thursday, September 6, 2018 14:36
To: burgesschen
Subject: Re: Increased Size of Incremental Checkpoint

Hi

I think the "checkpoint size" metrics showed in your graph means the total checkpoint size
of each time. The incremental checkpoint, take rocksDB state-backend as example, means during
each checkpoint's async-snapshot process, only the newly generated sst files would be uploaded
to HDFS, which reducing the pressure to network. Since your job continues to read message
at a stable rate, the total checkpoint size would continue to increase.

Best
Yun
________________________________
From: burgesschen <tchen325@bloomberg.net>
Sent: Thursday, September 6, 2018 4:29
To: user@flink.apache.org
Subject: Increased Size of Incremental Checkpoint

Hi guys,
I enabled incremental flink checkpoint for my flink job. I had the job read
messages at a stable rate. For each message, the flink job store something
in the keyed state. My question is: For every minute, the increased state
size is the same, shouldn't the incremental checkpoint size remain
relatively constant also? How come it is increasing as shown in the picture?
Thank you!


<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t969/incremental_check_point.png>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Mime
View raw message