ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-21580) Replace Hard Coded stack-select Structures
Date Mon, 31 Jul 2017 17:36:00 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107659#comment-16107659
] 

Hudson commented on AMBARI-21580:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7827 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7827/])
AMBARI-21580 - Replace Hard Coded stack-select Structures (jhurley: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=2bab2159ba9f17a0fcdefd20c8ab401c0042fdb9])
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_client.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/rest_api.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/zkfc_slave.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py
* (edit) ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py
* (edit) ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/drpc_server.py
* (edit) ambari-server/src/main/resources/common-services/PIG/0.16.1.3.0/package/scripts/pig_client.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/sqoop_client.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/mapreduce2_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py
* (edit) ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py
* (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/service_check.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py
* (edit) ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py
* (edit) ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/journalnode.py
* (edit) contrib/management-packs/odpi-ambari-mpack/src/main/resources/stacks/ODPi/2.0/hooks/after-INSTALL/scripts/shared_initialization.py
* (edit) ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/service_check.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py
* (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
* (add) ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_select_packages.json
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/application_timeline_server.py
* (edit) ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_admin.py
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py
* (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server_interactive.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_metastore.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py
* (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_client.py
* (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/package/scripts/master.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server.py
* (edit) ambari-server/src/test/python/stacks/utils/RMFTestCase.py
* (edit) ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_usersync.py
* (edit) ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/kafka_broker.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor_prod.py
* (edit) ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/datanode.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/job_history_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hcat_client.py
* (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_script.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus_prod.py
* (edit) ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py
* (edit) contrib/management-packs/odpi-ambari-mpack/src/main/resources/stacks/ODPi/2.0/hooks/after-INSTALL/scripts/hook.py
* (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/job_history_server.py
* (edit) contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/after-INSTALL/scripts/shared_initialization.py
* (edit) ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout_client.py
* (edit) ambari-server/src/main/resources/stacks/HDP/3.0/configuration/cluster-env.xml
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/nfsgateway.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py
* (edit) ambari-common/src/main/python/resource_management/libraries/script/script.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor_prod.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/pacemaker.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py
* (edit) ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/yarn_client.py
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/job_history_server.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/livy_server.py
* (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py
* (edit) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/hook.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/drpc_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py
* (edit) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_master.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
* (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/knox_gateway.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/snamenode.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
* (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py
* (edit) ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka_broker.py
* (add) ambari-server/src/main/resources/stacks/HDP/3.0/properties/stack_select_packages.json
* (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/superset.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/webhcat_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/pacemaker.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_regionserver.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py
* (edit) ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py
* (edit) contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/after-INSTALL/scripts/hook.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py
* (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py
* (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/metadata_server.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/historyserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/ui_server.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py
* (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py
* (edit) ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/FinalUpgradeCatalog.java
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/rest_api.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hcat_client.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_tagsync.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
* (edit) ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_tagsync.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus_prod.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py
* (edit) ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/atlas_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_server.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/resourcemanager.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py
* (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py


> Replace Hard Coded stack-select Structures
> ------------------------------------------
>
>                 Key: AMBARI-21580
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21580
>             Project: Ambari
>          Issue Type: Task
>    Affects Versions: 2.5.3
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 2.5.3
>
>
> Currently, both the stack-select and conf-select python files have hard-coded structures
inside of them which contain mappings for things like configuration directories and stack-select
component names. 
> This is mainly used for pairing the Ambari role name (defined from metainfo.xml) with
the {{stack-select}} package name. There are places, like {{params.py}} where we have not
yet entered the target python file (like {{AccumuloScript}}) where we could get this information.
Some components need this to be able to build things like the Hadoop conf dir:
> {code:title=stack_select.py}
> SERVER_ROLE_DIRECTORY_MAP = {
>   'ACCUMULO_MASTER' : 'accumulo-master',
>   'ACCUMULO_MONITOR' : 'accumulo-monitor',
>   'ACCUMULO_GC' : 'accumulo-gc',
>   'ACCUMULO_TRACER' : 'accumulo-tracer',
>   'ACCUMULO_TSERVER' : 'accumulo-tablet',
>   'ATLAS_SERVER' : 'atlas-server',
>   'FLUME_HANDLER' : 'flume-server',
>   'FALCON_SERVER' : 'falcon-server',
> ...
> {code}
> With the coming of management packs replacing stacks, we can no longer hard code this
in Python. My suggestion is to begin moving this data into a property of some sort. Today,
{{cluster-env}} exists and would be the only place for it (where the stack-feature tools are
now). However, I believe that {{cluster-env}} is also going away and being replaced with something
similar to "Cluster Settings". 
> In any event, this Jira is to track the work needed to replace this logic.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message