Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.apache.org
Content-Type: multipart/alternative;
 boundary="===============4030123340627634791=="
MIME-Version: 1.0
Subject: Re: Review Request 40698: FALCON-1512 Implement touch feature for
 native scheduler
From: "Pallavi Rao" <pallavi.rao@inmobi.com>
To: "pavan kumar kolamuri" <pavan.kolamuri@gmail.com>,
 "Falcon" <dev@falcon.incubator.apache.org>,
 "Pallavi Rao" <pallavi.rao@inmobi.com>
Date: Fri, 11 Dec 2015 10:46:53 -0000
Message-ID: <20151211104653.1618.4112@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Pallavi Rao" <noreply@reviews.apache.org>
References: <20151209095805.1618.52816@reviews.apache.org>
In-Reply-To: <20151209095805.1618.52816@reviews.apache.org>
Reply-To: "Pallavi Rao" <pallavi.rao@inmobi.com>

--===============4030123340627634791==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/DAGEngine.java, line 122
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153802#file1153802line122>
> >
> >     Pallavi can you add description in jira what touch will do in native scheduler

touch with native scheduler will do the same thing that it does now. Ensures any updates to libraries are picked up (without any change to entity).


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java, line 346
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153803#file1153803line346>
> >
> >     I didn't get what exactly touch is doing. We are just creating staging path.  Will it be useful when effective time come into picture ?

It just builds the entity againin a new staging path. This copies over the libraries to the new staging path afresh. That way, when the next instance is run, the latest libs get picked up.

When effective time comes into picture, we'll have to re-run older instances. Will cross that bridge when we get to it. :-)


> On Dec. 9, 2015, 9:58 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java, line 363
> > <https://reviews.apache.org/r/40698/diff/3/?file=1153803#file1153803line363>
> >
> >     As of now effective time is always in future (now + 3 mins). Won't it break backward compatability ?

now + 3 is more of a limitation because scheduling on Oozie takes time. With native scheduler, the very next instance that is run, will pick up the changes.


- Pallavi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40698/#review109478
-----------------------------------------------------------


On Dec. 4, 2015, 12:06 p.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40698/
> -----------------------------------------------------------
> 
> (Updated Dec. 4, 2015, 12:06 p.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1512
>     https://issues.apache.org/jira/browse/FALCON-1512
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Touch feature was not implemented for native scheduler. Adding it now.
> 
> 
> Diffs
> -----
> 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/DAGEngine.java ebc05ec 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java ac7cde8 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/OozieDAGEngine.java 70c8353 
>   scheduler/src/test/java/org/apache/falcon/execution/MockDAGEngine.java 087114f 
> 
> Diff: https://reviews.apache.org/r/40698/diff/
> 
> 
> Testing
> -------
> 
> Manual testing done. Couldn't add UTs are there are interactions with Oozie.
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


--===============4030123340627634791==--