falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Smith <lenov...@gmail.com>
Subject lifecycle - retention
Date Fri, 22 Jan 2016 17:55:16 GMT
Hello,

I found that Falcon supports retention policy as part of the Lifecycle. I
am wondering how is it working, because its not clear to me by reading the
documentation.

Assume I store one file  (with thousands/million of records) into HDFS and
I set retention period for 1 year.

How is that retention period enforced on the records inside the file? Does
it mean that scheduler executes some "flow" that reads record by record of
the stored file every day and check the current date agains retention date?
In case the current date >= retention date the record is removed. Is it
cpu/time consuming? Each check requires the full file scan?

What will happen in scenario when I define different retention dates per
field?



Thank you!

Best,
John

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message