falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh" <sram...@hortonworks.com>
Subject Re: Review Request 39711: Lifecycle does not allow feed with frequency greater than days(1)
Date Wed, 28 Oct 2015 23:46:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39711/#review104372
-----------------------------------------------------------



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 813)
<https://reviews.apache.org/r/39711/#comment162628>

    Sorry, for multiple comments. I didn't review Lifecycle feature so I didn't have the complete
picture.
    
    Frequency in the retention stage is not mandatory and if teh frequency is not set by user
then
    1> If feed frequency < 6 hrs its set to 6 hrs
    2> If its > 6 hrs its set to feed frequency
    
    Shouldn't it fallbaack to current behavior for retenting the data? < 6hrs set to 6
hrs and > 6hrs set to 1 day?
    
    This is required for 2 reasons
    1> Current understanding of users is that if feed frequency > 6 hrs , retention
job will run every day. We shouldn't deviate from this.
    
    2> I also spoke with Venkatesh about why was it set to 1 day. He mentioned in case
retention fails and reruns fail too we don't want to keep the data till it runs next time
if feed frequency is used. This can cause SEC retention vioalation and also cause memory issues
if feed frequency is say one year. If job runs every day it catches up for the scenario mentioned
above.
    
    Any specific reason to change the old behavior?


- Sowmya Ramesh


On Oct. 28, 2015, 6:04 p.m., Ajay Yadava wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39711/
> -----------------------------------------------------------
> 
> (Updated Oct. 28, 2015, 6:04 p.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1560
>     https://issues.apache.org/jira/browse/FALCON-1560
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Lifecycle does not allow feed with frequency greater than days(1)
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/FeedHelper.java 5c252a8 
>   common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java 4020d36 
>   common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java 905be68

> 
> Diff: https://reviews.apache.org/r/39711/diff/
> 
> 
> Testing
> -------
> 
> Added unit test for the scenarios.
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message