falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Re: Review Request 37520: FALCON-1231 Reliable notifications for Oozie workflow status
Date Wed, 23 Sep 2015 06:10:30 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37520/
-----------------------------------------------------------

(Updated Sept. 23, 2015, 6:10 a.m.)


Review request for Falcon.


Changes
-------

Incorporated the additional requirement of notifying on wait and timeout.


Bugs: FALCON-1231
    https://issues.apache.org/jira/browse/FALCON-1231


Repository: falcon-git


Description
-------

Workflow completion is currently dependent on post processing action putting messages (success/failure)
to a JMS queue.
When failures occur on Hadoop / Oozie, the post processing action may not get executed either.
In such cases, Falcon will be blind to the status of the workflow and re-tries will not happen.
Instead, use JMS notifications of Oozie to get job completion notifications.

The patch is backward compatible. It will handle notifications from post-processing and Oozie.
That way users are not forced to perform any migration steps, but, can move to Oozie notifications
in a phased manner.


Diffs (updated)
-----

  client/src/main/java/org/apache/falcon/resource/InstancesResult.java 76bb4b0 
  common/src/main/java/org/apache/falcon/catalog/CatalogPartitionHandler.java d0b09df 
  common/src/main/java/org/apache/falcon/entity/WorkflowNameBuilder.java 6890594 
  common/src/main/java/org/apache/falcon/metadata/MetadataMappingService.java ef9da45 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionArgs.java 9456fb9 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 4454239 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionListener.java 2d3a477 
  common/src/main/java/org/apache/falcon/workflow/WorkflowJobEndNotificationService.java c4f8843

  common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 0b560bb

  common/src/test/java/org/apache/falcon/entity/TestWorkflowNameBuilder.java 6060731 
  common/src/test/java/org/apache/falcon/workflow/WorkflowExecutionContextTest.java 65a057d

  common/src/test/java/org/apache/falcon/workflow/WorkflowJobEndNotificationServiceTest.java
b7df443 
  messaging/pom.xml 6528278 
  messaging/src/main/java/org/apache/falcon/messaging/JMSMessageConsumer.java d3178fb 
  messaging/src/test/java/org/apache/falcon/messaging/JMSMessageConsumerTest.java 5bbcc87

  oozie-el-extensions/src/main/conf/oozie-site.xml 0925b41 
  oozie/src/main/java/org/apache/falcon/workflow/FalconPostProcessing.java cff1187 
  oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 0441f7c 
  oozie/src/test/java/org/apache/falcon/oozie/workflow/FalconPostProcessingTest.java 9d31d17

  rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunHandler.java 785dce8 
  rerun/src/main/java/org/apache/falcon/rerun/handler/RetryHandler.java b952bbe 
  webapp/src/conf/oozie/conf/oozie-site.xml 8545ef9 

Diff: https://reviews.apache.org/r/37520/diff/


Testing
-------

UTs added. Manual testing done.


Thanks,

Pallavi Rao


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message