ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Speidel" <jspei...@hortonworks.com>
Subject Re: Review Request 29276: Implements support for deploying HDFS NameNode High-Availability (HA) clusters using a Blueprint
Date Sun, 21 Dec 2014 16:51:15 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29276/#review65744
-----------------------------------------------------------

Ship it!


Ship It!

- John Speidel


On Dec. 19, 2014, 11:11 p.m., Robert Nettleton wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29276/
> -----------------------------------------------------------
> 
> (Updated Dec. 19, 2014, 11:11 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, John Speidel, Robert Levas, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-8771
>     https://issues.apache.org/jira/browse/AMBARI-8771
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This patch resolves AMBARI-8771.
> 
> In previous releases of Ambari Blueprints, it was not possible to deploy an HDFS NameNode
HA cluster using a Blueprint.  The existing HA support in Ambari in the UI supported adding
HA support to a running cluster, but Ambari did not support
>  creating HA clusters from the initial deployment of the cluster.
> 
> This patch adds support for deploying HDFS NameNode HA clusters via the following changes:
> 
>   - Modifies the HDFS stack service scripts to handle the startup and
>     configuration requirements of an initial HDFS HA install, including
>     handling the startup of the active and standby nodes
>   - Implements support for two new configuration properties in hadoop-env
>     that allow the user to specify the initial states of the namenodes, by
>     declaring the host name of the active and standby nodes.  These
>     properties are used by the stack scripts in order to run the setup
>     specific to each node type.  If these properties are not set by
>     the user, the Blueprints processor will choose defaults for the
>     active and stanbdy nodes in the HDFS cluster.
>   - Updates the Blueprint configuration code to handle quorum journal URL
>     properties, which use a slightly different syntax than other Hadoop
>     configuration properties.
>   - Updates the Blueprint validation code to support a NameNode HA deployment
>     that does not include the SECONDARY_NAMENODE.  Currently, our stacks
>     require the secondary namenode, and there is no way to express a
>     cardinality that is optional based on HA status, so this handling
>     now occurs in the Blueprint processor.
>   - Adds/updates unit tests for all Java changes in this patch
>   - Adds/updates unit tests for all Python stack changes in this patch
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java
9cfb635 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
c4071d4 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
e8dbc59 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py
c9c81bb 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
ee8b418 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessorTest.java
8540d8b 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
abc0b15 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 9c65210 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_zkfc.py c358e9e 
>   ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_active_node.json PRE-CREATION

>   ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_standby_node.json PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/29276/diff/
> 
> 
> Testing
> -------
> 
> 1. Ran the ambari-server unit tests: "Tests run: 2451, Failures: 0, Errors: 0, Skipped:
13"
> 2. Manually verified that I can start a 3-node HDFS HA Cluster with a Blueprint. I used
the Ambari UI to verify that the cluster started via the Blueprint sets up the active and
standby nodes.  I've also used the "hdfs haadmin" tool to verify the status of the NameNodes
setup during a Blueprint-based HA install.  
> 3. Several times this week, I also started up a cluster with my changes applied, and
then used the UI Wizard to setup NameNode HA, to make sure the stack changes don't break the
UI support for adding HA to an existing cluster. 
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message