Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============3494301967266368881=="
MIME-Version: 1.0
Subject: Re: Review Request 41774: BE: Install packages call takes more than 3
 minutes on 900 node cluster
From: "Alejandro Fernandez" <afernandez@hortonworks.com>
To: "Jayush Luniya" <jluniya@hortonworks.com>,
 "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Yusaku Sako" <yusaku@hortonworks.com>,
 "Dmytro Grinenko" <hapy.lestat@gmail.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>,
 "Nate Cole" <ncole@hortonworks.com>
Cc: "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Ambari" <dev@ambari.apache.org>
Date: Tue, 29 Dec 2015 23:03:11 -0000
Message-ID: <20151229230311.4182.49526@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Alejandro Fernandez" <noreply@reviews.apache.org>
References: <20151229154445.4181.25549@reviews.apache.org>
In-Reply-To: <20151229154445.4181.25549@reviews.apache.org>
Reply-To: "Alejandro Fernandez" <afernandez@hortonworks.com>

--===============3494301967266368881==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41774/#review112244
-----------------------------------------------------------


What is the improvement with this?

- Alejandro Fernandez


On Dec. 29, 2015, 3:44 p.m., Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41774/
> -----------------------------------------------------------
> 
> (Updated Dec. 29, 2015, 3:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, Jayush Luniya, Nate Cole, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-14520
>     https://issues.apache.org/jira/browse/AMBARI-14520
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> On 900 host cluster the call to install packages took like 3 minutes 20 seconds which is kind of long.
> {noformat}
> POST http://perf-a-1:8080/api/v1/clusters/c1/stack_versions
> 
> {"ClusterStackVersions":{"stack":"HDP","version":"2.2","repository_version":"2.2.9.0-3393"}}:
> {noformat}
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java 6e7e06b 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java b6ac291 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java 43a4423 
>   ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java c56d7ce 
> 
> Diff: https://reviews.apache.org/r/41774/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> Current request execution time: 0m22.678s
> 
> With current patch, most time (about 40%) is consumed by findConfigurationTagsWithOverrides(cluster, hostname), but it seems to be inevitable even with batching addExecutionCommandsToStage() calls into a single invocation. I've tried inserting logging statements to verify that number of invocations equals to number of hosts.
> Rest of time is mostly consumed by org.apache.ambari.server.actionmanager.Stage.addHostRoleExecutionCommand() (20% of time) and by N invocations of ClustersImpl.getHostsForCluster() (19% of time). The latter calls can not be cached in method, so I can only rewrite method signature AmbariActionExecutionHelper.addExecutionCommandsToStage(ActionExecutionContext, Stage) and cache it in ClusterStackVersionResourceProvider. Seems to be a bad deal considering 1-3 seconds (on large clusters ) of performance boost for a rare call vs readability/maintainability of code.
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>


--===============3494301967266368881==--