ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Nettleton" <>
Subject Re: Review Request 34849: Adds property filtering support for the cluster update stage in Blueprints
Date Sun, 31 May 2015 01:02:50 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated May 31, 2015, 1:02 a.m.)

Review request for Ambari, Mahadev Konar, Robert Levas, and Sumit Mohanty.


Further testing with HDFS NameNode HA clusters revealed filtering out the HDFS NameNode (non-HA)
properties in an HA environment appears to break HDFS HA cluster startups with Blueprints.

I've updated the patch to remove the filter for HDFS Namenode properties, and left the HBase/Hive
filtering intact.  I'll re-introduce the HDFS filter in a separate patch once I've determined
the cause of the stack script failures I'm seeing, since the scripts seem to depend upon these
non-HA properties in HA mode.

Bugs: AMBARI-11563

Repository: ambari


This patch addresses AMBARI-11563. 

The Blueprints processor currently has no notion of "optional" properties in a cluster's configuration.
The Blueprints processor uses the stack definitions to determine the set of properties to
include and extend in the process of deploying a cluster.
There are some properties in HBase and Hive that are "conditional" in nature.

An example of this is the "hbase.coprocessor.regionserver.classes" config property in "hbase-site".
This property should only be set when HBase authorization is configured to be enabled.

Currently, a Blueprints deployment of HBase will set this property to the default, empty value,
and this will be included in the Cluster configuration. This causes problems after the cluster
deployment completes, since the Ambari UI will display this property as required, even through
authorization is not configured to be enabled in this cluster.
The Blueprints configuration processor needs to be modified to exclude any properties that
don't meet certain conditions:
1. HBase Authorization (on/off)
2. Hive Authentication (on/off)
3. HDFS NameNode properties that are only useful in a non-HA scenario must be filtered out
in an HDFS NameNode HA deployment.

This patch addresses this problem by:

1. Updating the Stack class used by Blueprints to read in the new property dependencies defined
in the stack
2. Extending the existing PropertyFilter interface in the BlueprintConfigurationProcessor,
such that it can now be used to filter out properties during the cluster startup phase, as
well as a Blueprint export.
3. Implemented filters to detect the HBase Authorization, Hive Authentication, and HDFS NameNode
HA cases described above. 
4. Adds unit tests to verify this change.

Diffs (updated)

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ 38b5a3c




1. Run the Blueprints-specific unit tests with this patch applied, all passing.  (I plan on
running the full ambari-server unit test suite prior to merging).
2. Deployed a single-node HBase and Hive cluster with authentication/authorization turned
off, and verified that the properties are not visible or required in the Ambari UI.  Verified
that the cluster REST resource reports the same results. 
3. Unit tests for these fixes cover the various cases (Hive Authentication on/off, HBase Authorization
on/off, etc).


Robert Nettleton

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message