ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Nettleton" <rnettle...@hortonworks.com>
Subject Review Request 29276: Implements support for deploying HDFS NameNode High-Availability (HA) clusters using a Blueprint
Date Fri, 19 Dec 2014 23:11:31 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29276/
-----------------------------------------------------------

Review request for Ambari, Jaimin Jetly, John Speidel, Robert Levas, and Yusaku Sako.


Bugs: AMBARI-8771
    https://issues.apache.org/jira/browse/AMBARI-8771


Repository: ambari


Description
-------

This patch resolves AMBARI-8771.

In previous releases of Ambari Blueprints, it was not possible to deploy an HDFS NameNode
HA cluster using a Blueprint.  The existing HA support in Ambari in the UI supported adding
HA support to a running cluster, but Ambari did not support
 creating HA clusters from the initial deployment of the cluster.

This patch adds support for deploying HDFS NameNode HA clusters via the following changes:

  - Modifies the HDFS stack service scripts to handle the startup and
    configuration requirements of an initial HDFS HA install, including
    handling the startup of the active and standby nodes
  - Implements support for two new configuration properties in hadoop-env
    that allow the user to specify the initial states of the namenodes, by
    declaring the host name of the active and standby nodes.  These
    properties are used by the stack scripts in order to run the setup
    specific to each node type.  If these properties are not set by
    the user, the Blueprints processor will choose defaults for the
    active and stanbdy nodes in the HDFS cluster.
  - Updates the Blueprint configuration code to handle quorum journal URL
    properties, which use a slightly different syntax than other Hadoop
    configuration properties.
  - Updates the Blueprint validation code to support a NameNode HA deployment
    that does not include the SECONDARY_NAMENODE.  Currently, our stacks
    require the secondary namenode, and there is no way to express a
    cardinality that is optional based on HA status, so this handling
    now occurs in the Blueprint processor.
  - Adds/updates unit tests for all Java changes in this patch
  - Adds/updates unit tests for all Python stack changes in this patch


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java
9cfb635 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
c4071d4 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
e8dbc59 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py
c9c81bb 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
ee8b418 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessorTest.java
8540d8b 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
abc0b15 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 9c65210 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_zkfc.py c358e9e 
  ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_active_node.json PRE-CREATION

  ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_standby_node.json PRE-CREATION


Diff: https://reviews.apache.org/r/29276/diff/


Testing
-------

1. Ran the ambari-server unit tests: "Tests run: 2451, Failures: 0, Errors: 0, Skipped: 13"
2. Manually verified that I can start a 3-node HDFS HA Cluster with a Blueprint. I used the
Ambari UI to verify that the cluster started via the Blueprint sets up the active and standby
nodes.  I've also used the "hdfs haadmin" tool to verify the status of the NameNodes setup
during a Blueprint-based HA install.  
3. Several times this week, I also started up a cluster with my changes applied, and then
used the UI Wizard to setup NameNode HA, to make sure the stack changes don't break the UI
support for adding HA to an existing cluster. 


Thanks,

Robert Nettleton


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message