falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava" <ajayn...@gmail.com>
Subject Re: Review Request 36538: Falcon Unit which supports Submit and Schedule of jobs (contributed by Pavan)
Date Sat, 18 Jul 2015 14:51:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36538/#review92171
-----------------------------------------------------------



common/src/main/java/org/apache/falcon/entity/EntityUtil.java (line 349)
<https://reviews.apache.org/r/36538/#comment146169>

    Will make more sense to put it in a DateUtil class. In future, we can move all date time
related methods to that class.



hadoop-dependencies/src/main/java/org/apache/falcon/hadoop/JailedFileSystem.java (line 61)
<https://reviews.apache.org/r/36538/#comment146170>

    We should add a falcon specific prefix to avoid accidental conflicts with existing directories
for some users.



oozie/src/main/java/org/apache/oozie/client/LocalOozieClientBundle.java (line 57)
<https://reviews.apache.org/r/36538/#comment146172>

    nit:
    for*
    without* protocol versioning



unit/src/main/java/org/apache/falcon/unit/FalconUnit.java (line 69)
<https://reviews.apache.org/r/36538/#comment146173>

    Should it be made thread safe if you want to ensure that start is not called twice?



unit/src/main/resources/deploy.properties (line 18)
<https://reviews.apache.org/r/36538/#comment146176>

    Will be good to add a comment explaining the purpose of this file and how it is used.
Will be very useful for someone new.


- Ajay Yadava


On July 16, 2015, 5:41 a.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36538/
> -----------------------------------------------------------
> 
> (Updated July 16, 2015, 5:41 a.m.)
> 
> 
> Review request for Falcon and Pallavi Rao.
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Submitted to Review Board on behalf of Pavan as he is on vacation for the 3 days.
> 
> Falcon unit will be able to help to validate the process, feed and cluster xml's and
as well as workflows of oozie. It will also help to submit schedule of jobs and check the
job status as well. Falcon Unit supports two modes 
> 1) Local Mode : It will bring up local oozie , use local file system and uses Local Job
Runner to run the jobs
> 2) Cluster Mode: Falcon unit Api's can also be run by pointing to external oozie, hdfs
and jobtracker.
> Documentation of falcon unit is provided in https://issues.apache.org/jira/browse/FALCON-1296
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/EntityUtil.java 63dfb9d 
>   common/src/main/java/org/apache/falcon/entity/parser/ClusterEntityParser.java 59b0910

>   common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java b5f531a

>   common/src/main/java/org/apache/falcon/workflow/util/OozieConstants.java PRE-CREATION

>   hadoop-dependencies/src/main/java/org/apache/falcon/hadoop/JailedFileSystem.java 7156bbd

>   oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java 03063f4 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieClientFactory.java 622238a

>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 4085b8f

>   oozie/src/main/java/org/apache/oozie/client/LocalOozieClientBundle.java PRE-CREATION

>   oozie/src/main/java/org/apache/oozie/client/LocalProxyOozieClient.java PRE-CREATION

>   pom.xml 31997e8 
>   unit/pom.xml PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnit.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitHelper.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalFalconClientProtocolProvider.java PRE-CREATION

>   unit/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
PRE-CREATION 
>   unit/src/main/resources/core-site.xml PRE-CREATION 
>   unit/src/main/resources/deploy.properties PRE-CREATION 
>   unit/src/main/resources/localoozie-log4j.properties PRE-CREATION 
>   unit/src/main/resources/log4j.xml PRE-CREATION 
>   unit/src/main/resources/mapred-site.xml PRE-CREATION 
>   unit/src/main/resources/oozie-site.xml PRE-CREATION 
>   unit/src/main/resources/startup.properties PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/examples/JavaExample.java PRE-CREATION 
>   unit/src/test/resources/cluster-template.xml PRE-CREATION 
>   unit/src/test/resources/infeed.xml PRE-CREATION 
>   unit/src/test/resources/input.txt PRE-CREATION 
>   unit/src/test/resources/outfeed.xml PRE-CREATION 
>   unit/src/test/resources/process.xml PRE-CREATION 
>   unit/src/test/resources/workflow.xml PRE-CREATION 
>   webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java 47b51fe

>   webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 6e58064

>   webapp/src/test/java/org/apache/falcon/util/OozieTestUtils.java 02d1011 
> 
> Diff: https://reviews.apache.org/r/36538/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message