flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: Clarification on state backend parameters
Date Fri, 03 Feb 2017 09:47:20 GMT

the purpose of the configuration parameter is described in the documentation under https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/config.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/config.html>. In
a nutshell, state.checkpoints.dir contains the (small) meta data files for checkpoints, which
typically contains pointers to the files which contain the actual state snapshot data. The
state.backend.fs.checkpointdir is the directory into which the actual state from the backends
is written. Finally, state.backend.rocksdb.checkpointdir is a poorly named key for the directory
of the RocksDB instance data and has in fact nothing to do with checkpoints.


> Am 03.02.2017 um 01:45 schrieb Mohit Anchlia <mohitanchlia@gmail.com>:
> Trying to understand these 3 parameters:
> state.backend
> state.backend.fs.checkpointdir
> state.backend.rocksdb.checkpointdir
> state.checkpoints.dir
> As I understand stream of data and the state of operators are 2 different concepts and
that both need to be checkpointed. I am bit confused about the purpose of these parameters
and their applicability.

View raw message