falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sandeep samudrala" <sandys...@gmail.com>
Subject Re: Review Request 39242: Improve Lifecycle Retention Validation checks
Date Tue, 13 Oct 2015 07:44:46 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39242/#review102399
-----------------------------------------------------------

Ship it!


Ship It!

- sandeep samudrala


On Oct. 13, 2015, 7:28 a.m., Ajay Yadava wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39242/
> -----------------------------------------------------------
> 
> (Updated Oct. 13, 2015, 7:28 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1524
>     https://issues.apache.org/jira/browse/FALCON-1524
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> 2 enhancements:
> 1. Check that retention should not be more frequent than the data availability.
> 2. Oozie doesn't behave appropriately for retention for high frequencies like minutes(3)
etc. Retention follows LAST_ONLY execution order and for high frequencies and default configuration
retention might not execute. To make it work you will need to override 
> oozie.service.CoordMaterializeTriggerService.materialization.window
> and
> oozie.service.CoordMaterializeTriggerService.lookup.interval
> which might affect the Oozie system performance.
> Hence we should set a limit like hours(1) for retention frequency and retention more
frequent than that should throw validation error.
> Originally reported by Pragya Mittal
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 6be2495

>   common/src/main/java/org/apache/falcon/lifecycle/retention/AgeBasedDelete.java a4ae780

>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java 1c43800

> 
> Diff: https://reviews.apache.org/r/39242/diff/
> 
> 
> Testing
> -------
> 
> Added Unit tests for all the scenarios.
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message