cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Mehta <>
Subject RE: API usage records and SQL
Date Thu, 31 May 2012 16:58:22 GMT
Hi Tamas,
Asking for 500 records takes 5-6 seconds and 1000 records taking 12 seconds shows a linear
increase and so doesn't seem like there is any performance degradation of any sort here.
>From a quick glance at the code it seems that your analysis  " It seems there is a conversion/parsing
algorithm somewhere in CS that struggles with the data " might be correct. 
I see that after fetching the data from DB, CS iterates over each record and makes a DB call
for setting the usage id (conversion from ID to UUID) depending on the usage type which
Indicates the DB table to look for. Example below

for (Object usageRecordGeneric : usageRecords) {
	if(usageRecord.getUsageType() == UsageTypes.RUNNING_VM || usageRecord.getUsageType() == UsageTypes.ALLOCATED_VM){
                	//VM Instance ID
                	usageRecResponse.setUsageId(identityDao.getIdentityUuid("vm_instance", usageRecord.getUsageId().toString()));
                } else if(usageRecord.getUsageType() == UsageTypes.IP_ADDRESS){
                    //IP Address ID
                } else if .......// So on for different usage type.

So for fetching 1000 records its making 1 +1000 Db queries. 


-----Original Message-----
From: Tamas Monos [] 
Sent: Thursday, May 31, 2012 7:50 PM
Subject: API usage records and SQL


I've spent lots of time trying to debug why does it take so long for the management server
to answer the: "listUsageRecords&pagesize=1000&page=X" query.
When I ask for 500 records I get results in about 5-6 seconds. If I increase the pagesize
to 1000 I need to wait 12 seconds to start getting any response.

This means to me somewhere there is a massive bottleneck either in mysql or the way cs generates
the pages for response.
I have tcpdumped and got proof that the sql server responds and delivers 1000 records in 300ms
and after the management "hesitates" for about 10 seconds before sends the respond down to
the client.
It seems there is a conversion/parsing algorithm somewhere in CS that struggles with the data.
I have and account with a month worth usage records (35k, 35 pages) and takes about 7 minutes
(408sec) to get the all the data via "listUsageRecords&pagesize=1000&page=X".
Every page adds a ~12 seconds delay (408/35=11.6). I have checked the server load and cpu
utilization but nothing. Java is around 10% of CPU and the load is 0.06.

This is just one account...
Any advice or suggestion is appreciated.


Tamas Monos                                               DDI         +44(0)2034687012
Chief Technical                                             Office    +44(0)2034687000
Veber: The Hosting Specialists               Fax         +44(0)871 522 7057<>

Follow us on Twitter:<>
Follow us on Facebook:<>

View raw message