ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <>
Subject [jira] [Commented] (AMBARI-11850) Blueprint export process is very slow in multi-node clusters
Date Thu, 11 Jun 2015 00:37:00 GMT


Hadoop QA commented on AMBARI-11850:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in ambari-server:


Test results:
Console output:

This message is automatically generated.

> Blueprint export process is very slow in multi-node clusters
> ------------------------------------------------------------
>                 Key: AMBARI-11850
>                 URL:
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.0.0
>            Reporter: Robert Nettleton
>            Assignee: Robert Nettleton
>            Priority: Critical
>             Fix For: 2.1.0
>         Attachments: AMBARI-11850.patch
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> The Blueprint export process (using the REST API to create a Blueprint based on a running
cluster) has been found to have some performance issues in multi-node clusters. 
> Steps to reproduce:
> 1. Setup a 3-node cluster. 
> 2. Using the Ambari UI, start a cluster that includes HDFS, Yarn, MapReduce, Zookeeper,
and Ambari Metrics.  Accept all the defaults in configuration and host assignment. 
> 3. When the cluster deployment succeeds, attempt to export a Blueprint with the following
> http://host:port/api/v1/clusters/clusterone?format=blueprint
> The Blueprint export will eventually succeed, but takes quite a while to complete.  With
some basic testing, I've determined that for a 3-node cluster, the Blueprint export process
can take between 7-16 seconds on average.  I've attempted the export multiple times to make
sure no initial startup issues were involved.
> The workaround for this issue is to turn off the Ambari Metrics services prior to exporting
the Blueprint.  When Metrics is turned off, the export is much faster.  
> The Blueprint export process needs to be updated, such that the Metrics and Alerting
services are not called by the REST framework during a Blueprint export.  These services add
quite a bit to the overall time it takes to export the Blueprint, but without any real benefit,
since the Blueprint does not make use of these services' data.  
> I'm working on a fix for this, and will be submitting a patch shortly.  

This message was sent by Atlassian JIRA

View raw message