ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emil Anca" <ea...@hortonworks.com>
Subject Re: Review Request 33848: Blueprints processor needs stronger validation of Blueprint JSON structure
Date Thu, 14 May 2015 08:51:45 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33848/
-----------------------------------------------------------

(Updated May 14, 2015, 8:51 a.m.)


Review request for Ambari, Costel Radulescu, John Speidel, Robert Levas, and Robert Nettleton.


Changes
-------

Added additional unit tests.


Bugs: AMBARI-10931
    https://issues.apache.org/jira/browse/AMBARI-10931


Repository: ambari


Description
-------

The Blueprints processor needs to have stronger validation during the process of handling
a POST for a newly-submitted Blueprint from a REST client.
The Blueprints structure requires that the "configurations" element be an array of Maps, with
each Map representing a given configuration type ("core-site", "storm-site", etc).
Typically, the configurations structure should look like:
""configurations": [
{
"mapred-env": {
"properties":
{ "mapreduce_log_dir_prefix": "/grid/0/log/hadoop-mapreduce" }
}
},
{ "mapred-site": {
"properties":
{ "install-test-mapred-site": "install-test-mapred-site-VALUE" }
}
},
{ "yarn-env": {
"properties":
{ "yarn_log_dir_prefix": "/grid/0/log/hadoop" }
}
}
]"
The snippet above is just an example to illustrate the expected format.
Recent changes to the Blueprint's structure to handle "properties_attributes" (such as "final")
have modified the structure of a Blueprint, so that configuration properties are now contained
in a "properties" map, internal to each configuration type. Creating new Blueprints using
this newer format (the older format is still acceptable to the processor) require that the
configuration types are still enclosed in a map, even though an internal map is also present,
which may be a source of confusion.
An example of an incorrect format would be:
""configurations": [
{
"mapred-env": {
"properties":
{ "mapreduce_log_dir_prefix": "/grid/0/log/hadoop-mapreduce" }
},
"mapred-site": {
"properties":
{ "install-test-mapred-site": "install-test-mapred-site-VALUE" }
},
"yarn-env": {
"properties":
{ "yarn_log_dir_prefix": "/grid/0/log/hadoop" }
}
]
"
Please note that in the example above a single map is created, with all configuration elements
being added to that entry's configuration type. This causes a corrupted Blueprint to be stored
and used for deployments, since all configuration elements added in the original Blueprint
will be registered under one of the config types used in the original Blueprint. This will
cause many types of cluster startup failures, as the configuration elements will be added
to incorrect locations.
When the Blueprint processor receives a POST submission with a new Blueprint that includes
this incorrect format for "configurations", the processor must reject this submission, and
send back a relevant error code/message, so that the user can resolve this problem prior to
attempting a cluster deployment.
This issue will not affect the older format for Blueprints, and so any backwards-compatible
Blueprints used should work fine. This issue will only occur when new Blueprints are submitted
that use the newer syntax/format for defining configuration properties (with support for properties_attributes)
as well.
This JIRA is being created to track this issue, as it could potentially be a user experience
issue going forward, and should probably be resolved in Ambari 2.1, if possible.


Problem: Blueprint "configurations" element structure is not validated which leads to cluster
creations error
Solution: The "configuration" element is checked for the correct structure as a List of Maps
containing a single configuration type


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
aab5395 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java
118a7be 

Diff: https://reviews.apache.org/r/33848/diff/


Testing
-------

mvn clean test


Thanks,

Emil Anca


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message