ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaimin Jetly" <jai...@hortonworks.com>
Subject Re: Review Request 28197: Falcon lineage needs changes to startup.properties
Date Wed, 19 Nov 2014 01:40:20 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28197/#review62102
-----------------------------------------------------------

Ship it!


Ship It!

- Jaimin Jetly


On Nov. 19, 2014, 1:36 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28197/
> -----------------------------------------------------------
> 
> (Updated Nov. 19, 2014, 1:36 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Mahadev Konar, Srimanth Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-8379
>     https://issues.apache.org/jira/browse/AMBARI-8379
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Summary of changes needed and implications
> 
> changes in falcon/conf/startup.properties:
> 
> 1) For Falcon lineage to work below changes are needed on Ambari deployed configs for
Falcon
> Implications : Lineage feature of Falcon is blocked without these changes
> 
> MetadataMappingService is required for lineage:
> ```
> *.application.services=org.apache.falcon.security.AuthenticationInitializationService,\
>                         org.apache.falcon.workflow.WorkflowJobEndNotificationService,\
>                         org.apache.falcon.service.ProcessSubscriberService,\
>                         org.apache.falcon.entity.store.ConfigurationStore,\
>                         org.apache.falcon.rerun.service.RetryService,\
> 			org.apache.falcon.rerun.service.LateRunService,\
> 		        org.apache.falcon.service.LogCleanupService,\
>                         org.apache.falcon.metadata.MetadataMappingService
> ```
> 
> ```
> ######### Graph Database Properties #########
> # Graph implementation
> *.falcon.graph.blueprints.graph=com.thinkaurelius.titan.core.TitanFactory
> 
> # Graph Storage
> *.falcon.graph.storage.directory=/grid/0/data/lineage/graphdb
> *.falcon.graph.storage.backend=berkeleyje
> *.falcon.graph.serialize.path=/grid/0/data/lineage
> *.falcon.graph.preserve.history=false
> ```
> 
> The properties that need a directory in the local file system should detect a mount point
that is available on the Falcon server, and the python scripts need to crease these directories
when the Falcon service is configured.
> 
> The folder permission are,
> owner:falcon user
> group:hadoop
> permissions:775
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/package/scripts/falcon.py
37dd213 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/package/scripts/params.py
2316326 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/configuration/falcon-startup.properties.xml
ec8b9c7 
>   ambari-web/app/data/HDP2/site_properties.js e9b8234 
>   ambari-web/app/models/service_config.js 255a679 
> 
> Diff: https://reviews.apache.org/r/28197/diff/
> 
> 
> Testing
> -------
> 
> Ran ambari-server unit tests, all of which passed.
> OK
> ----------------------------------------------------------------------
> Total run:664
> Total errors:0
> Total failures:0
> OK
> 
> Also used the Install Wizard to deploy an HDP 2.2 cluster with 2 nodes, one of which
was the Falcon server, and the other the client.
> The falcon-startup.properties contains the correct values, and the Falcon Service Check
passed.
> 
> http://c6411.ambari.apache.org:8080/api/v1/clusters/dev/configurations?type=falcon-startup.properties&tag=version1
> 
> 
> * *.application.services: "org.apache.falcon.security.AuthenticationInitializationService,\
org.apache.falcon.workflow.WorkflowJobEndNotificationService, \ org.apache.falcon.service.ProcessSubscriberService,\
org.apache.falcon.entity.store.ConfigurationStore,\ org.apache.falcon.rerun.service.RetryService,\
org.apache.falcon.rerun.service.LateRunService,\ org.apache.falcon.service.LogCleanupService,\
org.apache.falcon.metadata.MetadataMappingService",
> * *.falcon.graph.blueprints.graph: "com.thinkaurelius.titan.core.TitanFactory",
> * *.falcon.graph.preserve.history: "false",
> * *.falcon.graph.serialize.path: "/hadoop/falcon/data/lineage",
> * *.falcon.graph.storage.backend: "berkeleyje",
> * *.falcon.graph.storage.directory: "/hadoop/falcon/data/lineage/graphdb",
> 
> 
> Further, the permissions are correct,
> ls -la /hadoop/falcon/data/
> drwxrwxr-x 3 falcon hadoop 4096 Nov 18 23:54 lineage
> ls -la /hadoop/falcon/data/lineage/
> drwxrwxr-x 2 falcon hadoop 4096 Nov 19 00:03 graphdb
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message