falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-1524) Improve Lifecycle Retention validation checks
Date Fri, 09 Oct 2015 10:28:26 GMT
Ajay Yadava created FALCON-1524:
-----------------------------------

             Summary: Improve Lifecycle Retention validation checks
                 Key: FALCON-1524
                 URL: https://issues.apache.org/jira/browse/FALCON-1524
             Project: Falcon
          Issue Type: Improvement
            Reporter: Ajay Yadava
            Assignee: Ajay Yadava


2 enhancements:

1. Check that retention should not be more frequent than the data availability.
2. Oozie doesn't behave appropriately for retention for high frequencies like minutes(3) etc.
Retention follows LAST_ONLY execution order and for high frequencies and default configuration
retention might not execute. To make it work you will need to override 
oozie.service.CoordMaterializeTriggerService.materialization.window
and
oozie.service.CoordMaterializeTriggerService.lookup.interval
which might affect the Oozie system performance.
Hence we should set a limit like hours(1) for retention frequency and retention more frequent
than that should throw validation error.

Originally reported by [~pragya.mittal]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message