ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Levas" <rle...@hortonworks.com>
Subject Re: Review Request 35322: Blueprint export performance enhancements
Date Wed, 10 Jun 2015 21:56:06 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35322/#review87469
-----------------------------------------------------------

Ship it!


Ship It!

- Robert Levas


On June 10, 2015, 4:43 p.m., Robert Nettleton wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35322/
> -----------------------------------------------------------
> 
> (Updated June 10, 2015, 4:43 p.m.)
> 
> 
> Review request for Ambari, John Speidel, Mahadev Konar, Robert Levas, Sumit Mohanty,
and Yusaku Sako.
> 
> 
> Bugs: AMBARI-11850
>     https://issues.apache.org/jira/browse/AMBARI-11850
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This patch resolves the performance issues detailed in AMBARI-11850.  
> 
> The Blueprint export process was taking a long time, due to the fact that the PropertyProvider
instances associated with Ambari Metrics and Alerting always execute for the Cluster REST
resource.  In the case of a Blueprint export, the Cluster resource is accessed with a different
format, and a different renderer that creates the Blueprint based on the Cluster resource's
data.  The Blueprint process does not require the Metrics or Alerting data, so the extra processing
time is wasted for this particular case.  For very large clusters, this process can take over
30 minutes or more.  
> 
> The performance problems have been resolved by the following changes:
> 
> 1. Added a new method to the Renderer interface, so that custom renderer implementations
can indicate if the property provider support is needed.
> 2. The BaseRenderer's implementation of this method always returns true, to keep the
behavior for the rest of the renderers as it currently is, with the exception of the ClusterBlueprintRenderer.
 In the future, other renderers may be modified to take advantage of this optimization, depending
upon the functionality required.  
> 3. The ClusterBlueprintRenderer's implementation of this method always returns false.
 This indicates that the Blueprint renderer does not require the additional data provided
by AMS or Alerting during a Blueprint export.  Previously, the processing of this data was
happening regardless of the Renderer type.  
> 4. Updated the QueryImpl class to skip the call to ClusterController.populateResources()
if the Renderer does not require property provider supoprt.  This check occurs for resources
and any associated sub-resources. 
> 5. Adds unit tests to verify this change.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java 2319683

>   ambari-server/src/main/java/org/apache/ambari/server/api/query/render/BaseRenderer.java
7866aa4 
>   ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java
cfc9bc0 
>   ambari-server/src/main/java/org/apache/ambari/server/api/query/render/Renderer.java
f353d53 
>   ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java 01361d2

>   ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java
96abb8c 
>   ambari-server/src/test/java/org/apache/ambari/server/api/query/render/DefaultRendererTest.java
4981a67 
> 
> Diff: https://reviews.apache.org/r/35322/diff/
> 
> 
> Testing
> -------
> 
> 1. Ran the ambari-server unit tests (all passing). 
> 2. Ran basic timing tests against a Blueprint export with this patch applied.  Previously,
the export for a 3-node cluster was taking between 7-10 seconds.  With this new patch applied,
the Blueprint export takes roughly 50 milliseconds or less on average.  I used a timed "curl"
command to obtain this data. 
> 3. Took the Blueprint exported from the test in #2, and used this Blueprint to deploy
a brand new cluster, to verify that the Blueprint was valid.  The cluster deployment succeeded.
 I had to make a slight change to the Blueprint, to add command retry configuration, but this
is un-related to the performance changes.  The exported Blueprint will POST correctly without
any manual changes.  The cluster-env changes are just to active the new command retry feature
for multi-node Blueprint deployments.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message