ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-17573) Add atlas-application config sections to all services that run Atlas hook, e.g., Hive, Falcon, Storm, Sqoop
Date Wed, 06 Jul 2016 20:28:11 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-17573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365019#comment-15365019
] 

Alejandro Fernandez commented on AMBARI-17573:
----------------------------------------------

Need a re-review on addendum patches, [^AMBARI-17573.trunk.additional.patch] and [^AMBARI-17573.branch-2.4.additional.patch]

> Add atlas-application config sections to all services that run Atlas hook, e.g., Hive,
Falcon, Storm, Sqoop
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-17573
>                 URL: https://issues.apache.org/jira/browse/AMBARI-17573
>             Project: Ambari
>          Issue Type: Story
>          Components: stacks
>    Affects Versions: 2.4.0
>            Reporter: Alejandro Fernandez
>            Assignee: Alejandro Fernandez
>             Fix For: 2.4.0
>
>         Attachments: AMBARI-17573.branch-2.4.additional.patch, AMBARI-17573.branch-2.4.patch,
AMBARI-17573.trunk.additional.patch, AMBARI-17573.trunk.patch
>
>
> Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes reference atlas-application.properties
file from Atlas server config location - /etc/atlas/conf/atlas-application.properties.
> Not all properties in /etc/atlas/conf/atlas-application.properties are required in hooks
and some of these properties are sensitive enough not to expose them to hooks/clients.
> To address this concern:
>  1. atlas-application.properties should be added as a config section in each of the host
component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
>  2. These new config sections will only include properties that are required to the respective
hooks
>  3. During initial deployment, Ambari will initialize these properties with values in
Atlas server configuration.
> For each one of those services, create a config type called ${service}-atlas-application.properties
that will be saved to /etc/${service}/conf/application.properties
> These are the default values,
> *Falcon*
> atlas.hook.falcon.synchronous=false
> atlas.hook.falcon.numRetries=3
> atlas.hook.falcon.minThreads=5
> atlas.hook.falcon.maxThreads=5
> atlas.hook.falcon.keepAliveTime=10
> atlas.hook.falcon.queueSize
> *Storm*
> atlas.hook.storm.numRetries=3
> *Hive*
> atlas.hook.hive.synchronous=false
> atlas.hook.hive.numRetries=3
> atlas.hook.hive.minThreads=5
> atlas.hook.hive.maxThreads=5
> atlas.hook.hive.keepAliveTime=10
> atlas.hook.hive.queueSize=10000
> *Common for all hooks*
> atlas.kafka.zookeeper.connect=
> atlas.kafka.bootstrap.servers=
> atlas.kafka.zookeeper.session.timeout.ms=400
> atlas.kafka.zookeeper.connection.timeout.ms=200
> atlas.kafka.zookeeper.sync.time.ms=20
> atlas.kafka.hook.group.id=atlas
> atlas.notification.create.topics=true
> atlas.notification.replicas=1
> atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
> atlas.notification.kafka.service.principal=kafka/_HOST@EXAMPLE.COM
> atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
> atlas.jaas.KafkaClient.loginModuleName = com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.KafkaClient.loginModuleControlFlag = required
> atlas.jaas.KafkaClient.option.useKeyTab = true
> atlas.jaas.KafkaClient.option.storeKey = true
> atlas.jaas.KafkaClient.option.serviceName = kafka
> atlas.jaas.KafkaClient.option.keyTab = /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_HOST@EXAMPLE.COM
> atlas.cluster.name=



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message