incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamas Monos <tam...@veber.co.uk>
Subject RE: API usage records and SQL
Date Sat, 02 Jun 2012 12:08:58 GMT
Hi,

Thanks it worked!

-----Original Message-----
From: Sheng Yang [mailto:sheng@yasker.org] 
Sent: 01 June 2012 19:33
To: cloudstack-dev@incubator.apache.org
Subject: Re: API usage records and SQL

Hi Tamas,

I remember we got some trouble with github repo?

Please try the current official repo:

http://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git

Thanks!

-- Sheng

On Fri, Jun 1, 2012 at 11:19 AM, Tamas Monos <tamasm@veber.co.uk> wrote:
> Hi,
>
> I have used git to get the cloudstack source from github to have a look at this problem.
> I could not find that part of code so I run a "grep -R usageRecordGeneric ./*" in the
source folder and got no results.
> I can't find the code you've pasted earlier. Am I missing something?
>
> Regards
>
>
> -----Original Message-----
> From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> Sent: 31 May 2012 17:58
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: API usage records and SQL
>
> Hi Tamas,
> Asking for 500 records takes 5-6 seconds and 1000 records taking 12 seconds shows a linear
increase and so doesn't seem like there is any performance degradation of any sort here.
> From a quick glance at the code it seems that your analysis  " It seems there is a conversion/parsing
algorithm somewhere in CS that struggles with the data " might be correct.
> I see that after fetching the data from DB, CS iterates over each 
> record and makes a DB call for setting the usage id (conversion from 
> ID to UUID) depending on the usage type which Indicates the DB table 
> to look for. Example below
>
> for (Object usageRecordGeneric : usageRecords) {
>        if(usageRecord.getUsageType() == UsageTypes.RUNNING_VM || 
> usageRecord.getUsageType() == UsageTypes.ALLOCATED_VM){
>                        //VM Instance ID
>                        
> usageRecResponse.setUsageId(identityDao.getIdentityUuid("vm_instance", 
> usageRecord.getUsageId().toString()));
>                } else if(usageRecord.getUsageType() == 
> UsageTypes.IP_ADDRESS){
>                    //IP Address ID
>                    
> usageRecResponse.setUsageId(identityDao.getIdentityUuid("user_ip_addre
> ss", usageRecord.getUsageId().toString()));
>                } else if .......// So on for different usage type.
>
> So for fetching 1000 records its making 1 +1000 Db queries.
>
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Tamas Monos [mailto:tamasm@veber.co.uk]
> Sent: Thursday, May 31, 2012 7:50 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: API usage records and SQL
>
> Hi,
>
> I've spent lots of time trying to debug why does it take so long for the management server
to answer the: "listUsageRecords&pagesize=1000&page=X" query.
> When I ask for 500 records I get results in about 5-6 seconds. If I increase the pagesize
to 1000 I need to wait 12 seconds to start getting any response.
>
> This means to me somewhere there is a massive bottleneck either in mysql or the way cs
generates the pages for response.
> I have tcpdumped and got proof that the sql server responds and delivers 1000 records
in 300ms and after the management "hesitates" for about 10 seconds before sends the respond
down to the client.
> It seems there is a conversion/parsing algorithm somewhere in CS that struggles with
the data. I have and account with a month worth usage records (35k, 35 pages) and takes about
7 minutes (408sec) to get the all the data via "listUsageRecords&pagesize=1000&page=X".
> Every page adds a ~12 seconds delay (408/35=11.6). I have checked the server load and
cpu utilization but nothing. Java is around 10% of CPU and the load is 0.06.
>
> This is just one account...
> Any advice or suggestion is appreciated.
>
> Regards
>
> Tamas Monos                                               DDI
        
> +44(0)2034687012 Chief Technical                                    
        
> Office    +44(0)2034687000
> Veber: The Hosting Specialists               Fax         +44(0)871 522 
> 7057 http://www.veber.co.uk<http://www.veber.co.uk/>
>
> Follow us on Twitter: 
> www.twitter.com/veberhost<http://www.twitter.com/veberhost>
> Follow us on Facebook: 
> www.facebook.com/veberhost<http://www.facebook.com/veberhost>
>
>
>



Mime
View raw message