incubator-oozie-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OOZIE-591) Oozie continues to materialize new actions after end date modification
Date Thu, 05 Jan 2012 20:14:41 GMT

    [ https://issues.apache.org/jira/browse/OOZIE-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180796#comment-13180796
] 

jiraposter@reviews.apache.org commented on OOZIE-591:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3030/
-----------------------------------------------------------

(Updated 2012-01-05 20:12:58.753704)


Review request for oozie.


Changes
-------

Updated it based on the discussion with Mohammad and Alejandro 
Now the execution of the interrupt commands will take place in the XCommand instead of doing
in the Pollable Queue.

Thanks!
Mohamed


Summary
-------

Oozie continues to materialize new actions after end date modification. The main issue is
related how Oozie materialized the actions based on a FIFO priority queue. Changing the status
of a bundle/coordinator job should take higher priority than executing this job. 

The main idea is to enable the queue to handle interruptions. Mainly, once you finished what
you are working on currently, you should focus on executing this next action. 

The issue can be illustrated as follow: 

1. Configure a pipeline to run for 1 hour 
2. Start the pipeline 
3. After it starts materializing new actions, change the end time (in my example - to 10 minutes
after the pipeline 
starts) 
4. Monitor the coordinator apps - they will continue to materialize new actions past the end
time.


This addresses bug OOZIE-591.
    https://issues.apache.org/jira/browse/OOZIE-591


Diffs (updated)
-----

  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/Command.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/XCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStatusUpdateXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/command/wf/WfEndXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/service/RecoveryService.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/util/XCallable.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/TestCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionStartXCommand.java
1209829 
  http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/service/TestCallableQueueService.java
1209829 

Diff: https://reviews.apache.org/r/3030/diff


Testing
-------

Regression for all the Commands in addition for specific testing for the Interrupt Driven
Map


Thanks,

Mohamed


                
> Oozie continues to materialize new actions after end date modification
> ----------------------------------------------------------------------
>
>                 Key: OOZIE-591
>                 URL: https://issues.apache.org/jira/browse/OOZIE-591
>             Project: Oozie
>          Issue Type: New Feature
>            Reporter: Mohamed Battisha
>              Labels: patch
>         Attachments: Interrupt Driven Commands.docx
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Oozie continues to materialize new actions after end date modification. The main issue
is related how Oozie materialized the actions based on a FIFO priority queue. Changing the
status of a bundle/coordinator job should take higher priority than executing this job.
> The main idea is to enable the queue to handle interruptions. Mainly, once you finished
what you are working on currently, you should focus on executing this next action.
> The issue can be illustrated as follow:
> 1. Configure a pipeline to run for 1 hour
> 2. Start the pipeline
> 3. After it starts materializing new actions, change the end time (in my example - to
10 minutes after the pipeline
> starts)
> 4. Monitor the coordinator apps - they will continue to materialize new actions past
the end time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message