flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prashantnayak <prash...@intellifylearning.com>
Subject Re: S3 recovery and checkpoint directories exhibit explosive growth
Date Wed, 26 Jul 2017 02:31:03 GMT
Hi Stephan

Unclear on what you mean by the "trash" option... thought that was only
available for command line hadoop and not applicable for API, which is what
Flink uses?  If there is a configuration for the Flink/Hadoop connector,
please let me know.

Also, one additional thing about S3.... S3 supports this option of
"versioned" buckets... Since versioning basically results in a delete on a
object not actually deleting it (unless there is a bucket lifecycle
policy)... I think you should recommend that Flink users that rely on S3
turn off bucket versioning since it seems to not really be a factor for
Flink...

Thanks
Prashant



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/S3-recovery-and-checkpoint-directories-exhibit-explosive-growth-tp14270p14453.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message