incubator-ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Diachenko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-3085) API latency time increases linearly to number of hosts managed
Date Wed, 16 Oct 2013 16:47:42 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Oleksandr Diachenko updated AMBARI-3085:
----------------------------------------

    Attachment: AMBARI-3085.patch

During profiling server processing mentioned requests, I've found two methods with huge number
of invocations. - PropertyHelper.getPropertyCategory and PropertyHelper.containsArguments.
These methods performs string operations and returns results depending on propertyId. Therefore
I've added caching in them, which will cost us some additional memory(two HashMaps up to 10000
elements), but we will avoid same operations on string.

> API latency time increases linearly to number of hosts managed
> --------------------------------------------------------------
>
>                 Key: AMBARI-3085
>                 URL: https://issues.apache.org/jira/browse/AMBARI-3085
>             Project: Ambari
>          Issue Type: Task
>          Components: controller
>    Affects Versions: 1.4.0
>            Reporter: Srimanth Gunturi
>              Labels: performance
>         Attachments: AMBARI-3085.patch, Screen Shot 2013-09-03 at 4.56.47 PM.png
>
>
> Doing performance measurements on 10, 50 and 100 nodes, it was noticed that a variety
of API invocations have latency time increase linearly to number of hosts managed. 
> The specific API calls made were:
> /requests:
> {noformat}
> /api/v1/clusters/${cluster}/requests?to=end&page_size=10&fields=*,tasks/Tasks/*
> {noformat}
> /services:
> {noformat}
> /api/v1/clusters/${cluster}/services?fields=components/ServiceComponentInfo,components/host_components,components/host_components/HostRoles,components/host_components/metrics/jvm/memHeapUsedM,components/host_components/metrics/jvm/memHeapCommittedM,components/host_components/metrics/mapred/jobtracker/trackers_decommissioned,components/host_components/metrics/cpu/cpu_wio,components/host_components/metrics/rpc/RpcQueueTime_avg_time,components/host_components/metrics/flume/flume,components/host_components/metrics/yarn/Queue
> {noformat}
> /hosts:
> {noformat}
> /api/v1/clusters/${cluster}/hosts?fields=Hosts/host_name,Hosts/public_host_name,Hosts/disk_info,Hosts/cpu_count,Hosts/total_mem,Hosts/host_status,Hosts/last_heartbeat_time,Hosts/os_arch,Hosts/os_type,Hosts/ip,host_components,metrics/disk,metrics/load/load_one
> {noformat}
> Times of the latency involved in the calls in milliseconds is below:
> {noformat}
> /requests	10 hosts	50 hosts	100 hosts
> Total Time	117.93	482.1904762	903.5070423
> Latency Time	45.63	135.952381	239.6478873
> Transfer Time	72.3	346.2380952	663.8591549
> {noformat}
> {noformat}
> /services	10 hosts	50 hosts	100 hosts
> Total Time	627.1717172	1568.710145	2757.891892
> Latency Time	614.4848485	1514.202899	2670.594595
> Transfer Time	12.68686869	54.50724638	87.2972973
> {noformat}
> {noformat}
> /hosts	10 hosts	50 hosts	100 hosts
> Total Time	125.3505155	394.9402985	711.943662
> Latency Time	123.2886598	379.9701493	683.4366197
> Transfer Time	2.06185567	14.97014925	28.47540984
> {noformat}
> Graphs are attached.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message