ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Speidel" <jspei...@hortonworks.com>
Subject Re: Review Request 27250: Proper handling exporting of 0.0.0.0 addresses with Blueprints
Date Tue, 28 Oct 2014 00:31:36 GMT


> On Oct. 27, 2014, 10:58 p.m., Michael Harp wrote:
> > There are several other properties that we typically set to 0.0.0.0, should they
be added?
> >             "hbase.master.ipc.address": "0.0.0.0",
> >             "hbase.regionserver.ipc.address": "0.0.0.0"
> >             "dfs.namenode.rpc-bind-host": "0.0.0.0",
> >             "dfs.namenode.http-bind-host": "0.0.0.0",
> >             "dfs.namenode.servicerpc-bind-host": "0.0.0.0",
> >             "dfs.namenode.secondary.http-address": "0.0.0.0:50090"
> >             "yarn.resourcemanager.webapp.address": "0.0.0.0:8088",
> >             "yarn.nodemanager.webapp.bind-host": "0.0.0.0",
> >             "yarn.nodemanager.webapp.https.bind-host": "0.0.0.0",
> >             "yarn.resourcemanager.admin.bind-host": "0.0.0.0",
> >             "yarn.resourcemanager.bind-host": "0.0.0.0",
> >             "yarn.resourcemanager.resource-tracker.bind-host": "0.0.0.0",
> >             "yarn.resourcemanager.scheduler.bind-host": "0.0.0.0",
> >             "yarn.timeline-service.webapp.bind-host": "0.0.0.0",
> >             "yarn.timeline-service.webapp.https.bind-host": "0.0.0.0",
> >             "yarn.timeline-service.bind-host": "0.0.0.0",
> >             "yarn.timeline-service.address": "0.0.0.0:10200"
> >             "mapreduce.jobhistory.admin.bind-host": "0.0.0.0",
> >             "mapreduce.jobhistory.webapp.bind-host": "0.0.0.0",
> >             "mapreduce.jobhistory.webapp.https.bind-host": "0.0.0.0",
> >             "mapreduce.jobhistory.bind-host": "0.0.0.0"

Michael,

For this issue, some service properties were not being exported to blueprint because they
were known to contain topology related information (hostname/ip) and there was no host that
matched the host/ip in the cluster.  Because of this, the blueprint processor removed these
properties from the exported set as it thought that they were pointing to external non-managed
hosts.  This patch simply ensures that any of these topology related properties which have
a value of "0.0.0.0" will be exported to blueprint.

>From the set of properties that you have included above, only a few are included in any
Ambari HDP stack.  For these properties that are included in an Ambari HDP stack, this patch
will ensure that they are exported to blueprint.  For the others in the set, the vast majority,
they would not have been affected by this issue since they are not "known" to be topology
related properties so they wouldn't have be checked to see if they pointed to a managed host.

On a related matter, because of limitations in the currently available stack definition syntax,
the blueprint processor has explicit knowledge of which properties are topology related. 
These properties are automatically updated to reflect the concrete cluster topology during
cluster provisioning via a blueprint.  It is only these properties which are affected by this
issue.  The special knowledge baked into the blueprint processor for these properites will
be moved to the stack definitions where it belongs in a future release.  This will make it
a lot easier to add custom stack definitions.


- John


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27250/#review58711
-----------------------------------------------------------


On Oct. 27, 2014, 9:58 p.m., Robert Nettleton wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27250/
> -----------------------------------------------------------
> 
> (Updated Oct. 27, 2014, 9:58 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Nate Cole.
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This patch implements a fix for AMBARI-7988.
> 
> The Blueprints config processor does not currently allow service
>   config properties with the IP address of "0.0.0.0" to be
>   included in an exported Blueprint.
> 
> The following properties:
> 
> yarn.timeline-service.address,
> yarn.timeline-service.webapp.address,
> yarn.timeline-service.webapp.https.address
> 
> are not included in an exported Blueprint after a successful
>   cluster deployment the Yarn Application Timeline Service
>   enabled.
> 
> The problem occurs because the default values for these
>   properties is typically "0.0.0.0:portNumber".  This is
>   a special IP address that the current Blueprints processor
>   does no recognize as a valid service address.  Currently,
>   the processor will remove any properties that require
>   host topology information, but don't have any valid hostname
>   included in the property.
> 
> This patch resolves the problem by adding some handling
>   code to the Blueprint export processing.  If a property, such
>   as the ones listed above, uses the default "0.0.0.0" address, then
>   this property is now allowed in the exported blueprint, as this
>   is a valid server address, but is not directly tied to the specific
>   server topology of a given cluster.  If the user overrides these
>   properties, then the hostname substitution process/export process
>   will work as before.
> 
> This patch also implements a new unit test to verify this change.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
4183b9d 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
04c75a1 
> 
> Diff: https://reviews.apache.org/r/27250/diff/
> 
> 
> Testing
> -------
> 
> 1. Ran the ambari-server unit tests with this patch applied (both trunk and 1.7.0), and
all unit tests are passing with this change.
> 2. Manually verified the fix on 1.7.0. 
> 3. Manually verified the fix on trunk.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message