falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha GS" <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 31343: FALCON-703 Monitoring plugin that registers catalog partition
Date Mon, 09 Mar 2015 11:45:57 GMT


> On March 5, 2015, 8:56 a.m., Suhas  Vasu wrote:
> > common/src/main/java/org/apache/falcon/catalog/CatalogPartitionHandler.java, line
98
> > <https://reviews.apache.org/r/31343/diff/1/?file=873493#file873493line98>
> >
> >     may be i am wrong, will there be no scenario where feednames & feedpaths
may not be a 1-1 mapping ?
> 
> Pallavi Rao wrote:
>     Suhas,
>     I looked at ProcessExecutionCoordinatorBuilder where this variable is populated.
The order of feednames and feedpaths match. As an additional shield, we can check to make
sure the basePath of the output instance path matches the one in the feed definition. Makes
sense?

I think feed retention messages don't honour this


- Shwetha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31343/#review75306
-----------------------------------------------------------


On Feb. 24, 2015, 7:40 a.m., Shwetha GS wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31343/
> -----------------------------------------------------------
> 
> (Updated Feb. 24, 2015, 7:40 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-703
>     https://issues.apache.org/jira/browse/FALCON-703
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> Added JMS listener that registers partitions for filesystem based feeds. Adding the first
version
> 
> Pending:
> 1. verify existing UTs
> 2. Add more tests
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/catalog/AbstractCatalogService.java 9abdc93

>   common/src/main/java/org/apache/falcon/catalog/CatalogPartitionHandler.java PRE-CREATION

>   common/src/main/java/org/apache/falcon/catalog/HiveCatalogService.java 25a4a46 
>   common/src/main/java/org/apache/falcon/entity/CatalogStorage.java 59f558b 
>   common/src/main/java/org/apache/falcon/entity/FeedHelper.java ca31f95 
>   common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java fe93048 
>   common/src/main/java/org/apache/falcon/entity/common/FeedDataPath.java 6ededbb 
>   common/src/main/java/org/apache/falcon/expression/ExpressionHelper.java e04f046 
>   common/src/main/java/org/apache/falcon/util/FalconRadixUtils.java bbd73c7 
>   common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 8d69b9a

>   common/src/main/resources/startup.properties 433c2a8 
>   common/src/test/java/org/apache/falcon/entity/FeedDataPathTest.java c405556 
>   common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java f6994fc 
>   common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java 1667161 
>   hadoop-dependencies/src/versioned-src/v2/java/org/apache/hadoop/mapred/ClassicClientProtocolProvider.java
2167375 
>   oozie/src/main/java/org/apache/falcon/logging/LogProvider.java 2e5dffb 
>   oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java
7a87919 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java ce292bd

>   prism/src/main/java/org/apache/falcon/Main.java de9f657 
>   prism/src/main/java/org/apache/falcon/listener/ContextStartupListener.java 5e879b1

>   prism/src/main/java/org/apache/falcon/util/EmbeddedServer.java 788df58 
>   prism/src/main/java/org/apache/falcon/util/SecureEmbeddedServer.java 8e9ef52 
>   test-tools/hadoop-webapp/src/main/java/org/apache/falcon/listener/HadoopStartupListener.java
7bee592 
>   test-tools/hadoop-webapp/src/main/resources/mapred-site.xml cf297de 
>   test-tools/hadoop-webapp/src/main/resources/yarn-site.xml 658752b 
>   webapp/pom.xml 556b171 
>   webapp/src/conf/oozie/conf/hadoop-conf/core-site.xml a0f1e07 
>   webapp/src/conf/oozie/conf/hadoop-conf/mapred-site.xml 96574b7 
>   webapp/src/conf/oozie/conf/oozie-site.xml ded4873 
>   webapp/src/test/java/org/apache/falcon/catalog/CatalogPartitionHandlerIT.java PRE-CREATION

>   webapp/src/test/java/org/apache/falcon/catalog/HiveCatalogServiceIT.java 71616e9 
>   webapp/src/test/java/org/apache/falcon/util/HiveTestUtils.java 19274b9 
>   webapp/src/test/java/org/apache/falcon/util/OozieTestUtils.java e67fe2a 
>   webapp/src/test/resources/cluster-template.xml 16b7c8c 
>   webapp/src/test/resources/feed-template1.xml 456f7ce 
>   webapp/src/test/resources/feed-template2.xml d4901fa 
> 
> Diff: https://reviews.apache.org/r/31343/diff/
> 
> 
> Testing
> -------
> 
> Manual end to end test, added one end to end success case
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message