ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-8188) Improve Ambari deployment performance in cloud environments
Date Thu, 06 Nov 2014 18:45:33 GMT
Ivan Mitic created AMBARI-8188:
----------------------------------

             Summary: Improve Ambari deployment performance in cloud environments
                 Key: AMBARI-8188
                 URL: https://issues.apache.org/jira/browse/AMBARI-8188
             Project: Ambari
          Issue Type: Improvement
    Affects Versions: 2.0.0
            Reporter: Ivan Mitic


This is an umbrella Jira that tracks improvements in the Ambari cluster deployment performance,
mainly targeting cloud environments. What is special about cloud environments is that VM images
could have preinstalled RPM/DEB packages.

Based on experiments run on public Azure on Large VMs SKU, Azure Storage (WASB) as a default
File System, and HDP 2.1 stack:

Ambari cluster deployment took:
* Install services stage – ~1.5 minute in average
** Given that all components are already preinstalled during this step Ambari just replays
all  install scripts and skips over the actual RPM/DEB package installations.
* Start services stage – ~12 minutes in average
** During this step Ambari performs additional setup of components/services required before
the services can be started up. Example of such setup steps are Namenode format, Oozie and
Hive metastore setup, Oozie and WebHCat shared lib setup. 

Detailed breakdown of the service start stage:
* NameNode format and exit safemode – 1:15
* History Server state setup (permissions and ownership FileSystem operations) – 0:50
* Hive Metastore setup – 0:30
* Hive Server 2 state setup (permissions and ownership FileSystem operations) – 1:20
* Oozie server metastore and shared lib setup (unzip + copy many small blobs + other FS operations)
– 3:25
* WebHCat shared lib setup – 1:30
* Overall services startup and scheduling overhead – 2:00



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message