Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============7889238382647589277=="
MIME-Version: 1.0
Subject: Re: Review Request 39481: Basic StackAdvisor support for blueprint
 (configurations)
From: "Oliver Szabo" <oszabo@hortonworks.com>
To: "Sumit Mohanty" <smohanty@hortonworks.com>,
 "Robert Nettleton" <rnettleton@hortonworks.com>
Cc: "Ambari" <dev@ambari.apache.org>, "Oliver Szabo" <oszabo@hortonworks.com>
Date: Wed, 28 Oct 2015 17:47:27 -0000
Message-ID: <20151028174727.22461.72424@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Oliver Szabo" <noreply@reviews.apache.org>
References: <20151027151745.22461.38649@reviews.apache.org>
In-Reply-To: <20151027151745.22461.38649@reviews.apache.org>
Reply-To: "Oliver Szabo" <oszabo@hortonworks.com>

--===============7889238382647589277==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39481/
-----------------------------------------------------------

(Updated Oct. 28, 2015, 5:47 p.m.)


Review request for Ambari, Robert Nettleton and Sumit Mohanty.


Changes
-------

- fixed comment
- added configurations input for stack advisor (and other ones too, despite of they are all needed or not for configuration recommendation)


Bugs: AMBARI-13487
    https://issues.apache.org/jira/browse/AMBARI-13487


Repository: ambari


Description
-------

During update configuration for cluster creation, configurations are upgraded based on response of the StackAdvisor.
The advised configurations will be added to cluster topology, and used during BlueprintConfigurationProcessor#doUpdateForClusterCreate(). The advised configurations filtered out based on ConfigRecommendationStrategy, which can be set in the Cluster Creation Template, e.g.:
{
  "blueprint" : "multi-node-conf-yarn",
  "default_password" : "secret",
  "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
    "host_groups": [
        {
          "hosts": [
              { "fqdn": "c6402.ambari.apache.org" }
          ],
          "name": "master",
          "configurations" : [ ]
        },
        {
          "hosts": [
              { "fqdn": "c6403.ambari.apache.org" }
          ],
          "name": "slave_1",
          "configurations" : [ ]
        }
    ],
  "Clusters" : {"cluster_name":"cluster_name"}
}
Also tested with host count and predicate:
...
"host_count" : ..,
"host_predicate" : "...",
...
I added the configuration modifications BEFORE the basic updates happen in BlueprintConfigurationProcessor.
As I see stackAdvisor not supports host group scoped configurations, so as my changes. I made a new AdvisedConfiguration which has a Map field for properties, this field can be a Configuration object in the future (if property attributes will be supported..because they are not supported too)

question: is that the correct way to geather stack defaults in BlueprintConfigurationProcessor#doFilterStackDefaults() ?


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java 5bc0962 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 18fca8a 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java e2f132e 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 4a906b1 
  ambari-server/src/main/java/org/apache/ambari/server/topology/AdvisedConfiguration.java PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 1ebde17 
  ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java 284a913 
  ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 5b716ae 
  ambari-server/src/main/java/org/apache/ambari/server/topology/ConfigRecommendationStrategy.java PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 64be609 
  ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 34b72b6 
  ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 8eeb54c 

Diff: https://reviews.apache.org/r/39481/diff/


Testing
-------

Unit tests passed:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:30 h
[INFO] Finished at: 2015-10-20T16:23:54+00:00
[INFO] Final Memory: 48M/687M
[INFO] -----------------------------------------------------------------------

Functional tests: Manually
(added "config_recommendation_strategy" field to the cluster creation template: tried 4 variations: field is missing, and set the field to NEVER_APPLY, ALWAYS_APPLY and ONLY_STACK_DEFAULTS_APPLY)


Thanks,

Oliver Szabo


--===============7889238382647589277==--