flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ziyad Muhammed <mmzi...@gmail.com>
Subject Re: confusing RocksDBStateBackend parameters
Date Sat, 17 Jun 2017 22:45:41 GMT
Hi,

To set the rocksdb state, you have two options:

1. Set the default state of the flink cluster, using the below parameters
in flink-conf.yaml file

state.backend: rocksdb

state.backend.fs.checkpointdir: hdfs://namenode:40010/flink/checkpoints


2. Set a per job state backend (which overrides the default setting)

StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();

env.setStateBackend(new
RocksDBStateBackend(parameterTool.getRequired("checkpointDataUri")));

Here you specify the path (for example, an HDFS directory like
hdfs://namenode:40010/flink/checkpoints) as the checkpointDataURI.

Hope that helps.

Best
Ziyad

Best Regards
*Ziyad Muhammed Mohiyudheen *
407, Internationales Studienzentrum Berlin
Theodor-Heuss-Platz 5
14052 Berlin
*Ph: +49 176 6587 3343 <%2B49%20176%206587%203343>*
*Mail to*: *mmziyad@gmail.com <mmziyad@gmail.com>*

On Fri, Jun 16, 2017 at 8:20 PM, Bowen Li <bowen.li@offerupnow.com> wrote:

> Hello guys,
>       I've been trying to figure out differences among several parameters
> of RocksDBStateBackend. The confusing parameters are:
>
>       In flink-conf.yaml:
>       1. state.backend.fs.checkpointdir
>       2. state.backend.rocksdb.checkpointdir
>       3. state.checkpoints.dir
>
>       and
>        4. the param *'**checkpointDataUri**'* you pass in to
> RocksDBStateBackend
> constructor in`public RocksDBStateBackend(*URI checkpointDataUri*)`
>
>         This email thread
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Clarification-on-state-backend-parameters-td11419.html>
> explained the first three well. But what's the 4th one for? What's its
> difference from others? I'd appreciate your clarification.
>
>        Thanks very much!
> Bowen
>
>
>

Mime
View raw message