falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan" <srik...@hotmail.com>
Subject Re: Review Request 23844: FALCON-481 Simplify process parent workflow
Date Fri, 25 Jul 2014 11:15:43 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23844/#review48722
-----------------------------------------------------------



oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java
<https://reviews.apache.org/r/23844/#comment85478>

    The reason order is not important is because the inter-dependency between actions are
also wired up here. Does it make sense to just build the post processing actions here and
stitch them in the correct flow sequence in the calling function ?


- Srikanth Sundarrajan


On July 24, 2014, 11:51 a.m., shwethags wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23844/
> -----------------------------------------------------------
> 
> (Updated July 24, 2014, 11:51 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-481
>     https://issues.apache.org/jira/browse/FALCON-481
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Currently, process parent workflow contains 2 decision nodes which take up oozie processing
and oozie db. The refactoring creates customised workflows for fs/hcat, recordsize/without
recordsize, pig/oozie/hive workflow engine processes. It also defines pre-processing and post-processing
actions only once so that the definitions are consistent
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/EntityUtil.java 6f50829 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java 62d95fa 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 5f483f0 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java 1238b82 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java
2ef162b 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java
PRE-CREATION 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationCoordinatorBuilder.java
0b582ef 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationWorkflowBuilder.java
2537725 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionCoordinatorBuilder.java
ac38532 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionWorkflowBuilder.java
eee4fe9 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/HCatReplicationWorkflowBuilder.java
PRE-CREATION 
>   oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java
79a1883 
>   oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java
977d8c1 
>   oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java
29f601d 
>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java
e46ae6e 
>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java
2e3a5c1 
>   oozie/src/main/java/org/apache/falcon/util/OozieUtils.java 9e1c82d 
>   oozie/src/main/resources/workflow/falcon-table-export.hql 37fd1b7 
>   oozie/src/main/resources/workflow/falcon-table-import.hql 653d580 
>   oozie/src/main/resources/workflow/feed/falcon-table-export.hql PRE-CREATION 
>   oozie/src/main/resources/workflow/feed/falcon-table-import.hql PRE-CREATION 
>   oozie/src/main/resources/workflow/feed/fs-replication-workflow.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/feed/hcat-replication-workflow.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/feed/replication-action.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/feed/retention-workflow.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/post-process.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/pre-process.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/process-parent-workflow.xml 4a2495c 
>   oozie/src/main/resources/workflow/process/hive-action.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/process/oozie-action.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/process/parent-workflow.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/process/pig-action.xml PRE-CREATION 
>   oozie/src/main/resources/workflow/replication-workflow.xml 0748acf 
>   oozie/src/main/resources/workflow/retention-workflow.xml 5138865 
>   oozie/src/test/java/org/apache/falcon/oozie/feed/OozieFeedWorkflowBuilderTest.java
c99c36c 
>   oozie/src/test/java/org/apache/falcon/oozie/process/AbstractTestBase.java cc0c419 
>   oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java
3655af9 
>   src/main/examples/entity/hcat/hcat-out-feed.xml 26d1f18 
> 
> Diff: https://reviews.apache.org/r/23844/diff/
> 
> 
> Testing
> -------
> 
> UTs + end to end tests for fs usecases
> 
> Pending: hcat testing
> 
> 
> Thanks,
> 
> shwethags
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message