falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Re: Review Request 37520: FALCON-1231 Reliable notifications for Oozie workflow status
Date Wed, 23 Sep 2015 06:11:21 GMT


> On Aug. 27, 2015, 11:58 p.m., Ying Zheng wrote:
> > webapp/src/conf/oozie/conf/oozie-site.xml, line 544
> > <https://reviews.apache.org/r/37520/diff/2/?file=1042826#file1042826line544>
> >
> >     Should it be "oozie.jms.producr.connection.properties"? typo? Same for the other
oozie-site.xml

It might be a typo from oozie side.. The property works as specified.


- Pallavi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37520/#review96799
-----------------------------------------------------------


On Sept. 23, 2015, 6:10 a.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37520/
> -----------------------------------------------------------
> 
> (Updated Sept. 23, 2015, 6:10 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1231
>     https://issues.apache.org/jira/browse/FALCON-1231
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Workflow completion is currently dependent on post processing action putting messages
(success/failure) to a JMS queue.
> When failures occur on Hadoop / Oozie, the post processing action may not get executed
either. In such cases, Falcon will be blind to the status of the workflow and re-tries will
not happen.
> Instead, use JMS notifications of Oozie to get job completion notifications.
> 
> The patch is backward compatible. It will handle notifications from post-processing and
Oozie. That way users are not forced to perform any migration steps, but, can move to Oozie
notifications in a phased manner.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/resource/InstancesResult.java 76bb4b0 
>   common/src/main/java/org/apache/falcon/catalog/CatalogPartitionHandler.java d0b09df

>   common/src/main/java/org/apache/falcon/entity/WorkflowNameBuilder.java 6890594 
>   common/src/main/java/org/apache/falcon/metadata/MetadataMappingService.java ef9da45

>   common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionArgs.java 9456fb9

>   common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 4454239

>   common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionListener.java 2d3a477

>   common/src/main/java/org/apache/falcon/workflow/WorkflowJobEndNotificationService.java
c4f8843 
>   common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java
0b560bb 
>   common/src/test/java/org/apache/falcon/entity/TestWorkflowNameBuilder.java 6060731

>   common/src/test/java/org/apache/falcon/workflow/WorkflowExecutionContextTest.java 65a057d

>   common/src/test/java/org/apache/falcon/workflow/WorkflowJobEndNotificationServiceTest.java
b7df443 
>   messaging/pom.xml 6528278 
>   messaging/src/main/java/org/apache/falcon/messaging/JMSMessageConsumer.java d3178fb

>   messaging/src/test/java/org/apache/falcon/messaging/JMSMessageConsumerTest.java 5bbcc87

>   oozie-el-extensions/src/main/conf/oozie-site.xml 0925b41 
>   oozie/src/main/java/org/apache/falcon/workflow/FalconPostProcessing.java cff1187 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 0441f7c

>   oozie/src/test/java/org/apache/falcon/oozie/workflow/FalconPostProcessingTest.java
9d31d17 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunHandler.java 785dce8 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/RetryHandler.java b952bbe 
>   webapp/src/conf/oozie/conf/oozie-site.xml 8545ef9 
> 
> Diff: https://reviews.apache.org/r/37520/diff/
> 
> 
> Testing
> -------
> 
> UTs added. Manual testing done.
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message