falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Re: Review Request 41495: FALCON-1639 Implement update feature for native scheduler
Date Fri, 18 Dec 2015 04:36:02 GMT


> On Dec. 17, 2015, 11:48 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/execution/ProcessExecutor.java, line 484
> > <https://reviews.apache.org/r/41495/diff/2/?file=1168317#file1168317line484>
> >
> >     Shouldn't we register here as we are unregistering to ServiceRegistery in suspend
?

During resume, the instances are moved to the appropriate state. If they are supposed to be
scheduled, onSchedule will get invoked and registering happens there.


> On Dec. 17, 2015, 11:48 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/notification/service/impl/SchedulerService.java,
line 213
> > <https://reviews.apache.org/r/41495/diff/2/?file=1168318#file1168318line213>
> >
> >     Can you add comment about parallelInstances here why is it differnt from process
concurrency. In InstanceRunner we are getting process concurrency right ?

The reason for adding parallelInstances is this - The update kicks in for new instances, but,
old waiting instances are scheduled and it retrieves the parallelism for entity definition,
it will the "new" parallelism (if the user has updated it). Since there is no versioning of
entities yet, need to store what was the parallelism when that instance was created.

Will add a comment to that effect.


> On Dec. 17, 2015, 11:48 a.m., pavan kumar kolamuri wrote:
> > scheduler/src/main/java/org/apache/falcon/predicate/Predicate.java, line 180
> > <https://reviews.apache.org/r/41495/diff/2/?file=1168319#file1168319line180>
> >
> >     Same here. why it is added as perdicate, we can always get from process definition
right ?

Answered above. Will add a comment.


- Pallavi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41495/#review110944
-----------------------------------------------------------


On Dec. 17, 2015, 8:20 a.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41495/
> -----------------------------------------------------------
> 
> (Updated Dec. 17, 2015, 8:20 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1639
>     https://issues.apache.org/jira/browse/FALCON-1639
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Adding support entity update via native scheduler
> 
> 
> Diffs
> -----
> 
>   scheduler/src/main/java/org/apache/falcon/execution/EntityExecutor.java c9c0f42 
>   scheduler/src/main/java/org/apache/falcon/execution/NotificationHandler.java 2f68ddb

>   scheduler/src/main/java/org/apache/falcon/execution/ProcessExecutor.java e446069 
>   scheduler/src/main/java/org/apache/falcon/notification/service/impl/SchedulerService.java
fb11091 
>   scheduler/src/main/java/org/apache/falcon/predicate/Predicate.java 164fb0e 
>   scheduler/src/main/java/org/apache/falcon/state/InstanceState.java 7f2bda9 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java
c19cada 
>   scheduler/src/main/java/org/apache/falcon/workflow/engine/OozieDAGEngine.java a26eb77

>   scheduler/src/test/java/org/apache/falcon/execution/FalconExecutionServiceTest.java
0ddf895 
> 
> Diff: https://reviews.apache.org/r/41495/diff/
> 
> 
> Testing
> -------
> 
> UT added. Manual testing done.
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message