flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marchant, Hayden " <hayden.march...@citi.com>
Subject Using local FS for checkpoint
Date Thu, 31 Aug 2017 07:53:46 GMT
Whether I use RocksDB or FS State backends, if my requirements are to have fault-tolerance
and ability to recover with 'at-least once' semantics for my Flink job, is there still a valid
case for using a backing local FS for storing states? i.e. If a Flink Node is invalidated,
I would have thought that the only way it could recover (by re-starting the task on different
node), is if the state is stored in  a shared file system such as HDFS, S3 etc....

I am asking since I want to know if HDFS is a must have for my deployment.



View raw message