falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shwetha...@inmobi.com
Subject Re: Review Request 23483: FALCON-353 enable dry run feature of oozie for schedule and update
Date Tue, 22 Jul 2014 07:03:10 GMT


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > common/src/main/java/org/apache/falcon/entity/EntityUtil.java, line 593
> > <https://reviews.apache.org/r/23483/diff/1/?file=632249#file632249line593>
> >
> >     we should log the error, will help in debugging in case of issues.

will log


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > common/src/main/java/org/apache/falcon/update/UpdateHelper.java, line 201
> > <https://reviews.apache.org/r/23483/diff/1/?file=632250#file632250line201>
> >
> >     it should return false here

good catch. will fix


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java,
line 91
> > <https://reviews.apache.org/r/23483/diff/1/?file=632251#file632251line91>
> >
> >     Changing the return type won't cause backward compatibility issues later ?
> >     my concern is because this is a server call.

Its internal to falcon, not an externally exposed API. So, its not an issue


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java,
line 1238
> > <https://reviews.apache.org/r/23483/diff/1/?file=632265#file632265line1238>
> >
> >     Shouldn't it be clone rather than entity ?

yes, should be. will fix


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java,
line 1242
> > <https://reviews.apache.org/r/23483/diff/1/?file=632265#file632265line1242>
> >
> >     Shouldn't it be clone rather than entity ?
> >

the name won't change in clone. So, not an issue


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > oozie/src/test/java/org/apache/falcon/oozie/feed/OozieFeedWorkflowBuilderTest.java,
line 447
> > <https://reviews.apache.org/r/23483/diff/1/?file=632266#file632266line447>
> >
> >     isn't new Path(wfPath) redundant as wfPath is already new Path()

its new Path(wfPath + "/scripts"), not just new Path(wfPath). Haven't changed it


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java,
line 997
> > <https://reviews.apache.org/r/23483/diff/1/?file=632265#file632265line997>
> >
> >     problem if update fails just before updating the config store, it will go through
all the steps again on retry.

will fix


> On July 22, 2014, 6 a.m., Suhas  Vasu wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java,
line 354
> > <https://reviews.apache.org/r/23483/diff/1/?file=632265#file632265line354>
> >
> >     what if we get back a empty list for delete/kill ?

It means the entity is not scheduled and does nothing


- shwethags


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23483/#review48206
-----------------------------------------------------------


On July 15, 2014, 11:57 a.m., shwethags wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23483/
> -----------------------------------------------------------
> 
> (Updated July 15, 2014, 11:57 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-353
>     https://issues.apache.org/jira/browse/FALCON-353
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Entity schedule creates a bundle in oozie. But bundle submit is asynchronous and coord
create can fail and creates bad user experience. So, this patch uses dryRun feature of oozie
to validate the entity before scheduling in oozie.
> 
> DryRun is used in validate(), schedule() and update()
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/EntityUtil.java a38e553 
>   common/src/main/java/org/apache/falcon/update/UpdateHelper.java 7af77d0 
>   common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java
eedd81f 
>   common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java e532b24 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java 2018db2 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 5a29683 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java bb8dfcc 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java
ac78297 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 6917f4e 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationCoordinatorBuilder.java
3226cf2 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationWorkflowBuilder.java
00fab99 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionCoordinatorBuilder.java
4393c94 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionWorkflowBuilder.java
4a7f96b 
>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 86cea93

>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java
c87bc86 
>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java
0d9abdb 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 38be792

>   oozie/src/test/java/org/apache/falcon/oozie/feed/OozieFeedWorkflowBuilderTest.java
542634d 
>   oozie/src/test/java/org/apache/falcon/oozie/process/AbstractTestBase.java 54a2ea7 
>   oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java
5ceea75 
>   prism/src/main/java/org/apache/falcon/FalconWebException.java d552c07 
>   prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java c135470 
>   webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java e214392

>   webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseySmokeIT.java dceb2f2

>   webapp/src/test/java/org/apache/falcon/resource/TestContext.java bc79609 
> 
> Diff: https://reviews.apache.org/r/23483/diff/
> 
> 
> Testing
> -------
> 
> UTs + manual end to end test of feed retention and process schedule and update
> 
> 
> Thanks,
> 
> shwethags
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message