flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Clarification on state backend parameters
Date Fri, 03 Feb 2017 19:55:53 GMT
I thought rocksdb is used to as a store backend. If that is the case then
why would are there 2 configuration parameter? Or in other words what is
the behavior if both state.backend.fs.checkpointdir and
state.backend.rocksdb is set?

On Fri, Feb 3, 2017 at 1:47 AM, Stefan Richter <s.richter@data-artisans.com>

> Hi,
> the purpose of the configuration parameter is described in the
> documentation under https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/setup/config.html. In a nutshell, state.checkpoints.dir
> contains the (small) meta data files for checkpoints, which typically
> contains pointers to the files which contain the actual state snapshot
> data. The state.backend.fs.checkpointdir is the directory into which the
> actual state from the backends is written. Finally, state.backend.rocksdb.checkpointdir
> is a poorly named key for the directory of the RocksDB instance data and
> has in fact nothing to do with checkpoints.
> Best,
> Stefan
> Am 03.02.2017 um 01:45 schrieb Mohit Anchlia <mohitanchlia@gmail.com>:
> Trying to understand these 3 parameters:
> state.backend
> state.backend.fs.checkpointdir
> state.backend.rocksdb.checkpointdir
> state.checkpoints.dir
> As I understand stream of data and the state of operators are 2 different
> concepts and that both need to be checkpointed. I am bit confused about the
> purpose of these parameters and their applicability.

View raw message