cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koushik Das (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-4350) [Performance Testing] Adding hosts take much longer time than baselines
Date Fri, 16 Aug 2013 10:53:47 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742081#comment-13742081
] 

Koushik Das commented on CLOUDSTACK-4350:
-----------------------------------------

I even checked at the logs for all the 20K hosts (since they were huge, Sowmya shared them
privately with me), wanted to check the time for hosts that got added at the end (with ids
19K and above). There I found that the max. time taken for the agent associated with host
to come up is ~3 secs which is not a lot of time. Some snippets below with host ids > 19K

For simulated host SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0 time taken is ~2 secs.

2013-08-15 15:46:21,564 DEBUG [agent.transport.Request] (ApiServer-5:null) Seq -1--1: Startup
request from directly connected host:  { Cmd , MgmtId: -1, via: -1, Ver: v1, Flags: 11, [{"cpus":4,"speed":8000,"memory":8589934592,"dom0MinMemory":858993459,"poolSync":false,"vms":{},"caps":"hvm","hypervisorType":"Simulator","hostDetails":{"com.cloud.network.Networks.RouterPrivateIpStrategy":"DcGlobal"},"type":"Routing","dataCenter":"1","pod":"2000","cluster":"19992","guid":"8508fba2-512d-46cf-9a4b-9605f3fb32e0","name":"SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0","version":"4.2.0-SNAPSHOT","privateIpAddress":"172.8.216.142","privateMacAddress":"00:00:00:00:00:00","storageIpAddress":"172.8.216.142","storageMacAddress":"00:00:00:00:00:00","storageIpAddressDeux":"172.8.216.142","storageMacAddressDeux":"172.8.216.142","agentTag":"agent-simulator","wait":0},{"parent":"/mnt/bc666ff1-4d58-40a5-acf4-afa5d1c8fe68/","templateInfo":{},"totalSize":1099511627776,"poolInfo":{"uuid":"bc666ff1-4d58-40a5-acf4-afa5d1c8fe68","host":"172.8.216.142","localPath":"/mnt/bc666ff1-4d58-40a5-acf4-afa5d1c8fe68/","hostPath":"/mnt/bc666ff1-4d58-40a5-acf4-afa5d1c8fe68/","poolType":"Filesystem","capacityBytes":1099511627776,"availableBytes":0},"resourceType":"STORAGE_POOL","fsType":"Filesystem","hostDetails":{},"type":"Storage","guid":"8508fba2-512d-46cf-9a4b-9605f3fb32e0","wait":0}]
}
2013-08-15 15:46:21,589 DEBUG [cloud.resource.ResourceState] (ApiServer-5:null) Resource state
update: [id = 19993; name = SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0; old state
= Creating; event = InternalCreated; new state = Enabled]
2013-08-15 15:46:21,589 DEBUG [cloud.host.Status] (ApiServer-5:null) Transition:[Resource
state = Enabled, Agent event = AgentConnected, Host id = 19993, name = SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0]
2013-08-15 15:46:21,593 DEBUG [cloud.host.Status] (ApiServer-5:null) Agent status update:
[id = 19993; name = SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0; old status = Creating;
event = AgentConnected; new status = Connecting; old update count = 0; new update count =
1]
2013-08-15 15:46:23,657 DEBUG [cloud.host.Status] (ApiServer-5:null) Transition:[Resource
state = Enabled, Agent event = Ready, Host id = 19993, name = SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0]
2013-08-15 15:46:23,672 DEBUG [cloud.host.Status] (ApiServer-5:null) Agent status update:
[id = 19993; name = SimulatedAgent.8508fba2-512d-46cf-9a4b-9605f3fb32e0; old status = Connecting;
event = Ready; new status = Up; old update count = 1; new update count = 2]

For simulated host SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f time taken is ~3 secs.

2013-08-15 15:46:06,773 DEBUG [agent.transport.Request] (ApiServer-8:null) Seq -1--1: Startup
request from directly connected host:  { Cmd , MgmtId: -1, via: -1, Ver: v1, Flags: 11, [{"cpus":4,"speed":8000,"memory":8589934592,"dom0MinMemory":858993459,"poolSync":false,"vms":{},"caps":"hvm","hypervisorType":"Simulator","hostDetails":{"com.cloud.network.Networks.RouterPrivateIpStrategy":"DcGlobal"},"type":"Routing","dataCenter":"1","pod":"1999","cluster":"19986","guid":"1eb35d8b-8ba6-43aa-ace7-642d304ef85f","name":"SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f","version":"4.2.0-SNAPSHOT","privateIpAddress":"172.8.215.140","privateMacAddress":"00:00:00:00:00:00","storageIpAddress":"172.8.215.140","storageMacAddress":"00:00:00:00:00:00","storageIpAddressDeux":"172.8.215.140","storageMacAddressDeux":"172.8.215.140","agentTag":"agent-simulator","wait":0},{"parent":"/mnt/d2b3cd8a-2d7f-4c47-a3f7-725959c020ee/","templateInfo":{},"totalSize":1099511627776,"poolInfo":{"uuid":"d2b3cd8a-2d7f-4c47-a3f7-725959c020ee","host":"172.8.215.140","localPath":"/mnt/d2b3cd8a-2d7f-4c47-a3f7-725959c020ee/","hostPath":"/mnt/d2b3cd8a-2d7f-4c47-a3f7-725959c020ee/","poolType":"Filesystem","capacityBytes":1099511627776,"availableBytes":0},"resourceType":"STORAGE_POOL","fsType":"Filesystem","hostDetails":{},"type":"Storage","guid":"1eb35d8b-8ba6-43aa-ace7-642d304ef85f","wait":0}]
}
2013-08-15 15:46:06,813 DEBUG [cloud.resource.ResourceState] (ApiServer-8:null) Resource state
update: [id = 19987; name = SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f; old state
= Creating; event = InternalCreated; new state = Enabled]
2013-08-15 15:46:06,813 DEBUG [cloud.host.Status] (ApiServer-8:null) Transition:[Resource
state = Enabled, Agent event = AgentConnected, Host id = 19987, name = SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f]
2013-08-15 15:46:06,820 DEBUG [cloud.host.Status] (ApiServer-8:null) Agent status update:
[id = 19987; name = SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f; old status = Creating;
event = AgentConnected; new status = Connecting; old update count = 0; new update count =
1]
2013-08-15 15:46:09,959 DEBUG [cloud.host.Status] (ApiServer-8:null) Transition:[Resource
state = Enabled, Agent event = Ready, Host id = 19987, name = SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f]
2013-08-15 15:46:09,965 DEBUG [cloud.host.Status] (ApiServer-8:null) Agent status update:
[id = 19987; name = SimulatedAgent.1eb35d8b-8ba6-43aa-ace7-642d304ef85f; old status = Connecting;
event = Ready; new status = Up; old update count = 1; new update count = 2]


So as can be seen the time taken for agent to come up is not much even at 20K hosts.
                
> [Performance Testing] Adding hosts take much longer time than baselines
> -----------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4350
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4350
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>         Environment: 4.2, performance test env, with simulator
>            Reporter: Sowmya Krishnan
>            Assignee: Koushik Das
>            Priority: Critical
>              Labels: perfomance
>             Fix For: 4.2.0
>
>
> Performance test setup:
> Basic zone, 1 host/cluster, tyring to deploy 20K simulator Hosts, with host tags
> Compared to baseline numbers, deploying hosts is taking  much longer in the simulator
environment
> For the 1st 1000 hosts, it took about 4 mins to deploy as per baseline
> With 4.2, the 1st 1000 hosts are taking almost 7 minutes
> Configurations:
> heap size: -Xmx12288m
> db.cloud.url.params=prepStmtCacheSize=517&cachePrepStmts=true&prepStmtCacheSqlLimit=4096&includeInnodbStatusInDeadlockExceptions=true&logSlowQueries=true
> For 20K Hosts, the number exponentially increases and the deployment time also increases.
> Attaching trace logs of 4.2 for 1k simulator hosts deployment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message