ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumit Mohanty (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-14366) Create a script to allow config merge during Ambari takeover
Date Mon, 14 Dec 2015 14:16:46 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sumit Mohanty updated AMBARI-14366:
-----------------------------------
    Description: 
_Background_:

  * The scope of this task is to provide a script that reads a dir structure for all configs.
**For now only site.xml** since env files cannot be parsed, we might need to handle yaml files
in later version.
  * It produces a output of all the problems / conflicts found along with a configuration.json
which can be directly incorporated in a cluster **blueprint**

_Tasks_:

  * Please find the attached patch with a partial script that you can start from (Really basic
foundation patch).
  * Assume that user has copied all the configs in a directory, example:  
/tmp/my-master-configs/etc/hadoop/conf,  
/tmp/my-master-configs/etc/hbase/conf,  
/etc/my-slave-configs/hadoop/conf and so on. (There could be 5 core-site from
different master / slave / client)

  * Parse these files and merge them into a map of configs that can be exported to json based
on following rules: 
    * If propertyA is found in core-site1 and core-site2 with same value, add it to the map
    * If propertyA is found in core-site1 only, add it to the map
    * If propertyA is found in core-site1 and core-site2 with different value, add it to an
output file with same name as the config-type, as a conflict.  
Sample file (Choose appropriate formatting of choice) ::

        
                core-site-conflicts.txt ::
        || Conflicts ||
        | propertyA  | <path-to-config-file> | value |
        | propertyA  | <path-to-config-file> | value |
        

    * **Note**: Single run should generate all the conflicts, if no conflicts found generate
the json file
    * Provide ability to force output and also a verbose mode to log every step



  was:
_Background_:

  * Paypal 900 node cluster Ambari takeover needs to address the problem of host based configs
merged to create default Ambari configs.
  * The scope of this task is to provide a script that reads a dir structure for all configs.
**For now only site.xml** since env files cannot be parsed, we might need to handle yaml files
in later version.
  * It produces a output of all the problems / conflicts found along with a configuration.json
which can be directly incorporated in a cluster **blueprint**

_Tasks_:

  * Please find the attached patch with a partial script that you can start from (Really basic
foundation patch).
  * Assume that user has copied all the configs in a directory, example:  
/tmp/my-master-configs/etc/hadoop/conf,  
/tmp/my-master-configs/etc/hbase/conf,  
/etc/my-slave-configs/hadoop/conf and so on. (There could be 5 core-site from
different master / slave / client)

  * Parse these files and merge them into a map of configs that can be exported to json based
on following rules: 
    * If propertyA is found in core-site1 and core-site2 with same value, add it to the map
    * If propertyA is found in core-site1 only, add it to the map
    * If propertyA is found in core-site1 and core-site2 with different value, add it to an
output file with same name as the config-type, as a conflict.  
Sample file (Choose appropriate formatting of choice) ::

        
                core-site-conflicts.txt ::
        || Conflicts ||
        | propertyA  | <path-to-config-file> | value |
        | propertyA  | <path-to-config-file> | value |
        

    * **Note**: Single run should generate all the conflicts, if no conflicts found generate
the json file
    * Provide ability to force output and also a verbose mode to log every step




> Create a script to allow config merge during Ambari takeover
> ------------------------------------------------------------
>
>                 Key: AMBARI-14366
>                 URL: https://issues.apache.org/jira/browse/AMBARI-14366
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 2.2.1
>
>         Attachments: AMBARI-14366.patch
>
>
> _Background_:
>   * The scope of this task is to provide a script that reads a dir structure for all
configs. **For now only site.xml** since env files cannot be parsed, we might need to handle
yaml files in later version.
>   * It produces a output of all the problems / conflicts found along with a configuration.json
which can be directly incorporated in a cluster **blueprint**
> _Tasks_:
>   * Please find the attached patch with a partial script that you can start from (Really
basic foundation patch).
>   * Assume that user has copied all the configs in a directory, example:  
> /tmp/my-master-configs/etc/hadoop/conf,  
> /tmp/my-master-configs/etc/hbase/conf,  
> /etc/my-slave-configs/hadoop/conf and so on. (There could be 5 core-site from
> different master / slave / client)
>   * Parse these files and merge them into a map of configs that can be exported to json
based on following rules: 
>     * If propertyA is found in core-site1 and core-site2 with same value, add it to the
map
>     * If propertyA is found in core-site1 only, add it to the map
>     * If propertyA is found in core-site1 and core-site2 with different value, add it
to an output file with same name as the config-type, as a conflict.  
> Sample file (Choose appropriate formatting of choice) ::
>         
>                 core-site-conflicts.txt ::
>         || Conflicts ||
>         | propertyA  | <path-to-config-file> | value |
>         | propertyA  | <path-to-config-file> | value |
>         
>     * **Note**: Single run should generate all the conflicts, if no conflicts found generate
the json file
>     * Provide ability to force output and also a verbose mode to log every step



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message