falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh" <sram...@hortonworks.com>
Subject Re: Review Request 38082: Status of scheduled Process entity is shown as submitted in corner case
Date Wed, 09 Sep 2015 21:08:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38082/#review98267
-----------------------------------------------------------

Ship it!


Ship It!

- Sowmya Ramesh


On Sept. 3, 2015, 12:29 a.m., Balu Vellanki wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38082/
> -----------------------------------------------------------
> 
> (Updated Sept. 3, 2015, 12:29 a.m.)
> 
> 
> Review request for Falcon, Ajay Yadava, Sowmya Ramesh, and Venkat Ranganathan.
> 
> 
> Bugs: FALCON-1371
>     https://issues.apache.org/jira/browse/FALCON-1371
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Create a process entity with 
> - frequency of 1 hour
> - validity start as "2012-07-01T00:00Z" 
> - and validity end as "2012-07-01T01:02Z"
> 
> Now list the process entity after one hour. The process coordinator status in Oozie is
"succeeded". But Falcon shows the status as submitted.
> 
> 
> Diffs
> -----
> 
>   common/pom.xml 42facc9 
>   common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java
4d45cc7 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 7e6cd6c

>   prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java 78964dd 
> 
> Diff: https://reviews.apache.org/r/38082/diff/
> 
> 
> Testing
> -------
> 
> This patch was tested end2end for the following scenarios.
> 
> 1. When a bundle completed running successfully, status is now shown as "Succeeded" instead
of "Submitted"
> 
> 2. When the following feed is submitted, a coordinator is created for replication on
backupCluster, but no coordinator is created on primaryCluster for retention because the feed
validity on primaryCluster is in the past. There was a bug in code which was causing feed
status to be shown as "submitted" if any of the clusters have a MISSING bundle. After this
fix, the status will be shown as running.
> {code}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="cleansedEmailFeed" description="Cleansed customer emails" xmlns="uri:falcon:feed:0.1">
>     <groups>churnAnalysisDataPipeline</groups>
>     <frequency>hours(1)</frequency>
>     <timezone>UTC</timezone>
>     <late-arrival cut-off="hours(4)"/>
>     <clusters>
>         <cluster name="primaryCluster" type="source">
>             <validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
>             <retention limit="hours(90)" action="delete"/>
>             <locations>
>                 <location type="data" path="/user/ambari-qa/falcon/demo/primary/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
>                 <location type="stats" path="/"/>
>                 <location type="meta" path="/"/>
>             </locations>
>         </cluster>
>         <cluster name="backupCluster" type="target">
>             <validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
>             <retention limit="hours(90)" action="delete"/>
>             <locations>
>                 <location type="data" path="/falcon/demo/bcp/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
>                 <location type="stats" path="/"/>
>                 <location type="meta" path="/"/>
>             </locations>
>         </cluster>
>     </clusters>
>     <locations>
>         <location type="data" path="/user/ambari-qa/falcon/demo/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
>         <location type="stats" path="/"/>
>         <location type="meta" path="/"/>
>     </locations>
>     <ACL owner="ambari-qa" group="users" permission="0x755"/>
>     <schema location="/none" provider="/none"/>
> </feed>
> {code}
> 
> 3. If a bundle is killed using oozie command line, the status of entity will be now shown
as KILLED
> 
> 4. Added support for FAILED.
> 
> 5. Added methods to avoid calling findLatestBundle(...) repeatedly.
> 
> 
> Thanks,
> 
> Balu Vellanki
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message