falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Re: Review Request 40698: FALCON-1512 Implement touch feature for native scheduler
Date Fri, 11 Dec 2015 10:46:53 GMT


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/DAGEngine.java, line 122
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153802#file1153802line122>
> >
> >     Pallavi can you add description in jira what touch will do in native scheduler

touch with native scheduler will do the same thing that it does now. Ensures any updates to
libraries are picked up (without any change to entity).


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java,
line 346
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153803#file1153803line346>
> >
> >     I didn't get what exactly touch is doing. We are just creating staging path.
 Will it be useful when effective time come into picture ?

It just builds the entity againin a new staging path. This copies over the libraries to the
new staging path afresh. That way, when the next instance is run, the latest libs get picked
up.

When effective time comes into picture, we'll have to re-run older instances. Will cross that
bridge when we get to it. :-)


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java,
line 363
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153803#file1153803line363>
> >
> >     As of now effective time is always in future (now + 3 mins). Won't it break
backward compatability ?

now + 3 is more of a limitation because scheduling on Oozie takes time. With native scheduler,
the very next instance that is run, will pick up the changes.


- Pallavi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40698/#review109478
-----------------------------------------------------------


On Dec. 4, 2015, 12:06 p.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40698/
> -----------------------------------------------------------
> 
> (Updated Dec. 4, 2015, 12:06 p.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1512
>     https://issues.apache.org/jira/browse/FALCON-1512
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Touch feature was not implemented for native scheduler. Adding it now.
> 
> 
> Diffs
> -----
> 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/DAGEngine.java ebc05ec 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java
ac7cde8 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/OozieDAGEngine.java 70c8353

>   scheduler/src/test/java/org/apache/falcon/execution/MockDAGEngine.java 087114f 
> 
> Diff: https://reviews.apache.org/r/40698/diff/
> 
> 
> Testing
> -------
> 
> Manual testing done. Couldn't add UTs are there are interactions with Oozie.
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message