falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Review Request 37520: FALCON-1231 Reliable notifications for Oozie workflow status
Date Mon, 17 Aug 2015 11:34:29 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37520/
-----------------------------------------------------------

Review request for Falcon.


Bugs: FALCON-1231
    https://issues.apache.org/jira/browse/FALCON-1231


Repository: falcon-git


Description
-------

Workflow completion is currently dependent on post processing action putting messages (success/failure)
to a JMS queue.
When failures occur on Hadoop / Oozie, the post processing action may not get executed either.
In such cases, Falcon will be blind to the status of the workflow and re-tries will not happen.
Instead, use JMS notifications of Oozie to get job completion notifications.

The patch is backward compatible. It will handle notifications from post-processing and Oozie.
That way users are not forced to perform any migration steps, but, can move to Oozie notifications
in a phased manner.


Diffs
-----

  client/src/main/java/org/apache/falcon/resource/InstancesResult.java 76bb4b0 
  common/src/main/java/org/apache/falcon/catalog/CatalogPartitionHandler.java d0b09df 
  common/src/main/java/org/apache/falcon/entity/WorkflowNameBuilder.java 6890594 
  common/src/main/java/org/apache/falcon/metadata/MetadataMappingService.java ef9da45 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionArgs.java 9456fb9 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 53ef5de 
  common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionListener.java 2d3a477 
  common/src/main/java/org/apache/falcon/workflow/WorkflowJobEndNotificationService.java c4f8843

  common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 07fafb5

  common/src/test/java/org/apache/falcon/entity/TestWorkflowNameBuilder.java 6060731 
  common/src/test/java/org/apache/falcon/workflow/WorkflowExecutionContextTest.java 65a057d

  common/src/test/java/org/apache/falcon/workflow/WorkflowJobEndNotificationServiceTest.java
b7df443 
  messaging/pom.xml b222c35 
  messaging/src/main/java/org/apache/falcon/messaging/JMSMessageConsumer.java d3178fb 
  messaging/src/test/java/org/apache/falcon/messaging/JMSMessageConsumerTest.java 9d49b25

  oozie-el-extensions/src/main/conf/oozie-site.xml 0925b41 
  oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 2f3dc6f 
  rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunHandler.java c2cb09e 
  rerun/src/main/java/org/apache/falcon/rerun/handler/RetryHandler.java c6bc36f 
  webapp/src/conf/oozie/conf/oozie-site.xml ded4873 

Diff: https://reviews.apache.org/r/37520/diff/


Testing
-------

UTs added. Manual testing done.


Thanks,

Pallavi Rao


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message