ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaimin D Jetly (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-18055) service config page load takes long time on cluster with large number of service config versions
Date Mon, 08 Aug 2016 07:50:20 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jaimin D Jetly updated AMBARI-18055:
------------------------------------
    Summary: service config page load takes long time on cluster with large number of service
config versions  (was: config page load takes long time on cluster with large number of service
config versions)

> service config page load takes long time on cluster with large number of service config
versions
> ------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-18055
>                 URL: https://issues.apache.org/jira/browse/AMBARI-18055
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.4.0
>            Reporter: Jaimin D Jetly
>            Assignee: Jaimin D Jetly
>            Priority: Critical
>             Fix For: 2.4.0
>
>         Attachments: AMBARI-18055.patch, AMBARI-18055.trunk.patch
>
>
> It was observed that service config page load time took very long on clusters with 1000
and 4000 config versions. This was mainly because of slowness of below two APIs that are called
everytime when user navigates to any service config page
> *1. Get call to get the current service config version for a service along with other
dependent services:*
> {code}
> curl --user admin:admin -i  -H "X-Requested-By: ambari" -X GET "http://localhost:8080/api/v1/clusters/c1/configurations/service_config_versions?service_name.in(HIVE,ATLAS,YARN,TEZ,ZOOKEEPER,RANGER,HDFS)&is_current=true&fields=*"
> {code}
> Above API takes 7-8 seconds when any one of the service in the API has 1000 service config
versions. As a fix to this issue, Patch uses another namedQuery that precisely queries for
only current service config versions of the service rather than all service config versions
and then filtering the active ones in the code. With the patch it takes around 100-300ms for
the API to complete
> *2. Get call to get metadata for all service config version of a service:*
> {code}
> http://localhost:8080/api/v1/clusters/c1/configurations/service_config_versions?service_name=HDFS&fields=service_config_version,user,hosts,group_id,group_name,is_current,createtime,service_name,service_config_version_note,stack_id,is_cluster_compatible&minimal_response=true
> {code}
> It was analyzed that responses of all APIs are sorted by ambari-server either by the
sorting parameter specified by user in the API and if not explicitly specifed by the user
then default comparator is used for sorting. When default comparator is used, sorting result
set takes much longer time. Sorting 1000 service config version took around 6 seconds at [code|
https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterControllerImpl.java#L204].
As a workaround in the patch, I have changed this API to use sorting by appending "&sortBy=createtime.desc"
to the API. After this the API time came down from 6.5 seconds to 500ms for 1000 service config
versions. I have created AMBARI-18059 to investigate and work further on this issue. I have
also verified that any hosts or service config version API being used by ambari-web that can
return large result set uses sortBy parameter in the API. 
> Also created AMBARI-18058 for ambari-web to use pagination for this API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message