flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bowen Li <bowen...@offerupnow.com>
Subject Re: confusing RocksDBStateBackend parameters
Date Sun, 18 Jun 2017 07:04:31 GMT
Thanks for your clarification, Ziyad! I will try it out.

On Sat, Jun 17, 2017 at 3:45 PM, Ziyad Muhammed <mmziyad@gmail.com> wrote:

> Hi,
>
> To set the rocksdb state, you have two options:
>
> 1. Set the default state of the flink cluster, using the below parameters
> in flink-conf.yaml file
>
> state.backend: rocksdb
>
> state.backend.fs.checkpointdir: hdfs://namenode:40010/flink/checkpoints
>
>
> 2. Set a per job state backend (which overrides the default setting)
>
> StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
>
> env.setStateBackend(new RocksDBStateBackend(parameterTool.getRequired("checkpointDataUri")));
>
> Here you specify the path (for example, an HDFS directory like
> hdfs://namenode:40010/flink/checkpoints) as the checkpointDataURI.
>
> Hope that helps.
>
> Best
> Ziyad
>
> Best Regards
> *Ziyad Muhammed Mohiyudheen *
> 407, Internationales Studienzentrum Berlin
> Theodor-Heuss-Platz 5
> 14052 Berlin
> *Ph: +49 176 6587 3343 <%2B49%20176%206587%203343>*
> *Mail to*: *mmziyad@gmail.com <mmziyad@gmail.com>*
>
> On Fri, Jun 16, 2017 at 8:20 PM, Bowen Li <bowen.li@offerupnow.com> wrote:
>
>> Hello guys,
>>       I've been trying to figure out differences among several parameters
>> of RocksDBStateBackend. The confusing parameters are:
>>
>>       In flink-conf.yaml:
>>       1. state.backend.fs.checkpointdir
>>       2. state.backend.rocksdb.checkpointdir
>>       3. state.checkpoints.dir
>>
>>       and
>>        4. the param *'**checkpointDataUri**'* you pass in to
>> RocksDBStateBackend
>> constructor in`public RocksDBStateBackend(*URI checkpointDataUri*)`
>>
>>         This email thread
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Clarification-on-state-backend-parameters-td11419.html>
>> explained the first three well. But what's the 4th one for? What's its
>> difference from others? I'd appreciate your clarification.
>>
>>        Thanks very much!
>> Bowen
>>
>>
>>
>
>

Mime
View raw message