falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh" <sram...@hortonworks.com>
Subject Re: Review Request 41926: Retry does not update the state of the instance in the database
Date Wed, 06 Jan 2016 18:58:00 GMT


> On Jan. 5, 2016, 10:09 p.m., Balu Vellanki wrote:
> > rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunHandler.java, line
72
> > <https://reviews.apache.org/r/41926/diff/2/?file=1181893#file1181893line72>
> >
> >     If entity is defined, the wfEngine should be the workflow endpoint defined in
the cluster of this entity. If for any reason getting this wfEngine fails, shouldnt we propagate
the error up to the user? Please help me understand why we ignore the exception here...
> 
> pavan kumar kolamuri wrote:
>     Previously we always used to return the wfengine based on property defined in startup.properties
(workflow.engine.impl). With Native Scheduler even though this property is oozieworkflow engine,
there is chance that process is scheduled using native scheduler, in that case we should return
workflow engine based on what workflow engine it was scheduled. If it fails while getting
wfengine based on entity it should return default workflow engine defined in props which is
happening previously , thats why for backward compatibility we should always return default
workflow engine.

I don't fully understand the backward compatibility part here. Intention here is going forward
entities should be scheduled using Native scheduler even if user doesn't pass the scheduler
option in the cmd line? And if there was any error instanting the Native scheduler class then
fall back to default wf engine?

If so shouldn't this be done in WorkflowEngineFactory.getWorkflowEngine(entity) and not only
here.


Also lets say this particular entity was scheduled the first time using native scheduler and
it failed. Now for rerun if getting wfengine fails and default wf engine is returned wouldn't
job execution fail?


- Sowmya


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41926/#review112939
-----------------------------------------------------------


On Jan. 5, 2016, 12:42 p.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41926/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2016, 12:42 p.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1719
>     https://issues.apache.org/jira/browse/FALCON-1719
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Retry is not updating the state of the instance in the databases
> 
> 
> Diffs
> -----
> 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunConsumer.java 582cb15

>   rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunHandler.java 64c566e

>   rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java ee31952

>   rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunHandler.java 64177a4 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/RetryConsumer.java 61aa3e1 
> 
> Diff: https://reviews.apache.org/r/41926/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message