flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan <email2liy...@gmail.com>
Subject is there ways to enable checkpoint from flink-conf.yaml?
Date Wed, 26 Jul 2017 14:15:51 GMT
Hi , Flink users

we are using Flink as the runtime of our beam jobs which works great, 
recently we want to enable restart strategy 
<https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/restart_strategies.html>

in our flink cluster, from the document I see restart strategy will only 
work when checkpointing is enabled. I'm trying to find out if it's 
possible to enable checkpointing from flink-conf.yaml which is 
equivalent to the call " 
flinkStreamEnv.enableCheckpointing(checkpointInterval);" in 
StreamExecutionEnvironment.

the reason we want to config it through flink-conf.yaml is that we use 
helm to create flink cluster ondemand for dedicated job which works 
great on kubernetes env. with beam if we want to enable checkpointing , 
we have to create FlinkPipelineOptions which is cross platform. (like  
you use Hibernate Session in JPA code). so we are trying to find a way 
to enable it from flink-conf.yaml .

sample flink-conf.yaml as below.

   flink-conf.yaml: |
     blob.server.port: 6124
     jobmanager.rpc.address: address-cache-flink-jobmanager
     jobmanager.rpc.port: 6123
     jobmanager.heap.mb: 256
     taskmanager.heap.mb: 756
     taskmanager.numberOfTaskSlots: 4
     parallelism.default: 16
     metrics.reporters: prom
     metrics.reporter.prom.class: 
org.apache.flink.metrics.prometheus.PrometheusReporter
     metrics.reporter.prom.port: 9100-9101
     metrics.reporter.prom.prefix: flink_jm_
     restart-strategy: fixed-delay
     restart-strategy.fixed-delay.attempts: 3
     restart-strategy.fixed-delay.delay: 300 s
     state.backend: filesystem
     state.backend.fs.checkpointdir: 
file:///var/nfs/ephem_store/flink/checkpoints


Mime
View raw message