falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Smith <lenov...@gmail.com>
Subject lifecycle - retention
Date Fri, 22 Jan 2016 17:55:16 GMT

I found that Falcon supports retention policy as part of the Lifecycle. I
am wondering how is it working, because its not clear to me by reading the

Assume I store one file  (with thousands/million of records) into HDFS and
I set retention period for 1 year.

How is that retention period enforced on the records inside the file? Does
it mean that scheduler executes some "flow" that reads record by record of
the stored file every day and check the current date agains retention date?
In case the current date >= retention date the record is removed. Is it
cpu/time consuming? Each check requires the full file scan?

What will happen in scenario when I define different retention dates per

Thank you!


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message