cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jevgeni Zolotarjov <j.zolotar...@gmail.com>
Subject Re: Unable to create a deployment for VM
Date Fri, 20 Oct 2017 15:01:30 GMT
true, it does.

But for understanding the problem, I would have to
1. analyze the logs,
2. multiply [service offering core count] x [service offering MHz] and
3. check it against the remainder from the indicator on the dashboard.
Yes, it works that way too. But then I also have to
4. consider cpu.overprovisioning.factor (as I learned from you)

Well, I guess everybody would expect to see meaningful error message from
mature product. So indeed, I would like to contribute and at least post
this feature request.
Does someone know, how that works?

Best regards,
Jevgeni

On Fri, Oct 20, 2017 at 4:14 PM, Dag Sonstebo <Dag.Sonstebo@shapeblue.com>
wrote:

> Agree the error could be clearer – but if you check your CloudStack
> dashboard the pie chart for CPU already tells you your CPU utilisation.
>
> You can also use the listCapacity API call mentioned before:
>
> # cloudmonkey list capacity
> count = 9
> capacity:
> +--------------+--------------------------------------+-----
> --------+----------------+------+---------------------------+
> | capacityused |                zoneid                | percentused |
> capacitytotal  | type |          zonename         |
> +--------------+--------------------------------------+-----
> --------+----------------+------+---------------------------+
> |     6792     | d4b9d32e-d779-48b8-814d-d7847d55a684 |    31.51    |
>  21552      |  1   zonenamehere |
> | 332262574592 | d4b9d32e-d779-48b8-814d-d7847d55a684 |     1.11    |
> 30056223080448 |  3   | zonenamehere |
> |      17      | d4b9d32e-d779-48b8-814d-d7847d55a684 |      85     |
>    20       |  4   | zonenamehere |
> |      2       | d4b9d32e-d779-48b8-814d-d7847d55a684 |      10     |
>    20       |  5   zonenamehere |
> |      6       | d4b9d32e-d779-48b8-814d-d7847d55a684 |      30     |
>    20       |  7   | zonenamehere |
> |  5234491392  | d4b9d32e-d779-48b8-814d-d7847d55a684 |    35.24    |
> 14854279424   |  0   | zonenamehere |
> | 261059772416 | d4b9d32e-d779-48b8-814d-d7847d55a684 |     3.47    |
> 7514055770112  |  6   | zonenamehere |
> | 522119544832 | d4b9d32e-d779-48b8-814d-d7847d55a684 |     3.47    |
> 15028111540224 |  2   | zonenamehere |
> |      0       | d4b9d32e-d779-48b8-814d-d7847d55a684 |      0      |
>    0        |  19  | zonenamehere |
> +--------------+--------------------------------------+-----
> --------+----------------+------+---------------------------+
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 20/10/2017, 13:33, "Jevgeni Zolotarjov" <j.zolotarjov@gmail.com> wrote:
>
>     Thanks a lot!
>
>     It was it!
>
>     Actually the error reporting could be done in a wiser way to inform
>     administrator about what is the problem in particular. Do you know,
> where
>     can I request this feature?
>
>     Regards,
>     Jevgeni
>
>     On Fri, Oct 20, 2017 at 1:25 PM, Dag Sonstebo <
> Dag.Sonstebo@shapeblue.com>
>     wrote:
>
>     > Capacity type 1 is CPU: http://cloudstack.apache.org/
>     > api/apidocs-4.10/apis/listCapacity.html
>     >
>     > You would look at changing cluster.cpu.allocated.
> capacity.disablethreshold
>     > in global settings – or the same under your cluster settings.
>     > You can also play with cpu.overprovisioning.factor – but keep in mind
>     > overprovisioning is a science in itself – which has lots of gotcha’s
>     > depending on your workload.
>     >
>     > Regards,
>     > Dag Sonstebo
>     > Cloud Architect
>     > ShapeBlue
>     >
>     > On 20/10/2017, 11:16, "Jevgeni Zolotarjov" <j.zolotarjov@gmail.com>
> wrote:
>     >
>     >     I changed cluster.threshold.enabled to false. But it didn't help.
>     >     I guess the root cause it something different.
>     >
>     >     My storage capacity if above 3TB, and only under 300GB used.
>     >
>     >     No threshold setting was changed ever since installation.
>     >
>     >     On Fri, Oct 20, 2017 at 12:59 PM, Ivan Kudryavtsev <
>     > kudryavtsev_ia@bw-sw.com
>     >     > wrote:
>     >
>     >     > Just look for threshold. Logs don't include such information.
>     >     >
>     >     > 2017-10-20 16:56 GMT+07:00 Jevgeni Zolotarjov <
>     > j.zolotarjov@gmail.com>:
>     >     >
>     >     > > What is exact parameter name?
>     >     > >
>     >     > > On Fri, Oct 20, 2017 at 12:44 PM, Ivan Kudryavtsev <
>     >     > > kudryavtsev_ia@bw-sw.com
>     >     > > > wrote:
>     >     > >
>     >     > > > Ah, OK
>     >     > > >
>     >     > > > 2017-10-20 09:10:53,697 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > (API-Job-Executor-40:ctx-a4cff151 job-181 ctx-91a5717c)
>     >     > (logid:25e762a0)
>     >     > > > Cannot allocate cluster list [1] for vm creation since
> their
>     > allocated
>     >     > > > percentage crosses the disable capacity threshold defined
> at each
>     >     > > cluster/
>     >     > > > at global value for capacity Type : 1, skipping these
> clusters
>     >     > > >
>     >     > > > That's it. You have to change global variables or cluster
>     > configuration
>     >     > > > variables, because you're close to thresholds.
>     >     > > >
>     >     > > > 2017-10-20 16:40 GMT+07:00 Jevgeni Zolotarjov <
>     > j.zolotarjov@gmail.com
>     >     > >:
>     >     > > >
>     >     > > > > Hi,
>     >     > > > >
>     >     > > > > Yes I have one host. But I have exactly zero affinity
> groups.
>     > I could
>     >     > > > > successfully add few instances and they are running.
>     >     > > > > The
>     >     > > > >
>     >     > > > >
>     >     > > > > On Fri, Oct 20, 2017 at 12:35 PM, Ivan Kudryavtsev <
>     >     > > > > kudryavtsev_ia@bw-sw.com
>     >     > > > > > wrote:
>     >     > > > >
>     >     > > > > > Hi,
>     >     > > > > > It seems you have one host and place several VMs in a
> single
>     >     > > > > anti-affinity
>     >     > > > > > group, don't you? This is how it should really work if
> you
>     > do that
>     >     > > > thing.
>     >     > > > > > Anti-affinity groups are meaningless if you don't have
>     > several
>     >     > hosts.
>     >     > > > > >
>     >     > > > > > 2017-10-20 16:19 GMT+07:00 Jevgeni Zolotarjov <
>     >     > > j.zolotarjov@gmail.com
>     >     > > > >:
>     >     > > > > >
>     >     > > > > > > I could successfully create and run an instance with
>     > prepared
>     >     > > Compute
>     >     > > > > > > offering. But when I try to add one more instance of
> the
>     > same
>     >     > > Compute
>     >     > > > > > > offering, I am getting an error.
>     >     > > > > > > Unable to create a deployment for VM
>     >     > > > > > >
>     >     > > > > > > There are plenty of resources on host: cores, RAM,
> storage.
>     >     > > > > > >
>     >     > > > > > > Pleas help
>     >     > > > > > >
>     >     > > > > > > management-server.log:
>     >     > > > > > > 2017-10-20 09:10:43,713 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-3:null) (logid:) SeqA 5-377872:
>     > Processing
>     >     > > Seq
>     >     > > > > > > 5-377872:  { Cmd , MgmtId: -1, via: 5, Ver: v1,
> Flags: 11,
>     >     > > > > > > [{"com.cloud.agent.api.
> ConsoleProxyLoadReportCommand"
>     >     > > > > > > :{"_proxyVmId":1,"_loadInfo":"{\n
>     >     > > > > > > \"connections\": []\n}","wait":0}}] }
>     >     > > > > > > 2017-10-20 09:10:43,715 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-3:null) (logid:) SeqA 5-377872:
>     > Sending Seq
>     >     > > > > > > 5-377872:  { Ans: , MgmtId: 264216221068220, via: 5,
> Ver:
>     > v1,
>     >     > > Flags:
>     >     > > > > > > 100010,
>     >     > > > > > > [{"com.cloud.agent.api.
> AgentControlAnswer":{"result":
>     >     > > > true,"wait":0}}]
>     >     > > > > }
>     >     > > > > > > 2017-10-20 09:10:45,239 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-566e7818)
> (logid:24eee607)
>     > Begin
>     >     > > > cleanup
>     >     > > > > > > expired async-jobs
>     >     > > > > > > 2017-10-20 09:10:45,243 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-566e7818)
> (logid:24eee607)
>     > End
>     >     > > cleanup
>     >     > > > > > > expired
>     >     > > > > > > async-jobs
>     >     > > > > > > 2017-10-20 09:10:53,566 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc) (logid:41db964f)
> ===START===
>     >     > > > > > 192.168.1.194
>     >     > > > > > > -- GET
>     >     > > > > > > command=deployVirtualMachine&
>     > response=json&zoneid=b3d5b8bb-
>     >     > > > > > > 2dea-478b-8077-fc813b90d4c6&
> templateid=244d6029-0f9a-4479-
>     >     > > > > > > b11b-09e4fe6c40e1&hypervisor=KVM&serviceofferingid=
>     >     > > > > > > e0ef2da9-a489-4c42-ab6d-c49fd41b0fd2&diskofferingid=
>     >     > > > > > > f39fb02b-4fc0-4f2e-9e4e-dd54f7fc9508&size=50&
>     >     > > > > securitygroupids=1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe&
> displayname=mtl-dev04-app02&
>     >     > > > > > > name=mtl-dev04-app02&keyboard=us&_=1508490653550
>     >     > > > > > > 2017-10-20 09:10:53,581 DEBUG
> [c.c.u.AccountManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > Access
>     >     > > > > > granted
>     >     > > > > > > to Acct[15863393-8e8d-11e7-8f52-f04da2002bbe-admin]
> to
>     >     > > > > > > org.apache.cloudstack.quota.vo.ServiceOfferingVO$$
>     >     > > > > > > EnhancerByCGLIB$$4d79c9fa@540d8fcb
>     >     > > > > > > by AffinityGroupAccessChecker
>     >     > > > > > > 2017-10-20 09:10:53,581 DEBUG
> [c.c.u.AccountManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > Access
>     >     > > > > > granted
>     >     > > > > > > to Acct[15863393-8e8d-11e7-8f52-f04da2002bbe-admin]
> to
>     >     > > > > > > com.cloud.storage.DiskOfferingVO$$
>     > EnhancerByCGLIB$$f64346ed@
>     >     > > 7d0dcb96
>     >     > > > > by
>     >     > > > > > > AffinityGroupAccessChecker
>     >     > > > > > > 2017-10-20 09:10:53,605 DEBUG
> [c.c.v.UserVmManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocating
>     >     > > > > > in
>     >     > > > > > > the DB for vm
>     >     > > > > > > 2017-10-20 09:10:53,613 DEBUG [c.c.v.
>     > VirtualMachineManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocating
>     >     > > > > > > entries for VM: VM[User|i-2-23-VM]
>     >     > > > > > > 2017-10-20 09:10:53,614 DEBUG [c.c.v.
>     > VirtualMachineManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocating
>     >     > > > > > > nics for VM[User|i-2-23-VM]
>     >     > > > > > > 2017-10-20 09:10:53,614 DEBUG [o.a.c.e.o.
>     > NetworkOrchestrator]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocating
>     >     > > > > > nic
>     >     > > > > > > for vm VM[User|i-2-23-VM] in network
> Ntwk[204|Guest|6] with
>     >     > > requested
>     >     > > > > > > profile NicProfile[0-0-null-null-null
>     >     > > > > > > 2017-10-20 09:10:53,619 DEBUG [c.c.v.
>     > VirtualMachineManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocating
>     >     > > > > > > disks for VM[User|i-2-23-VM]
>     >     > > > > > > 2017-10-20 09:10:53,626 DEBUG [c.c.v.
>     > VirtualMachineManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > Allocation
>     >     > > > > > > completed for VM: VM[User|i-2-23-VM]
>     >     > > > > > > 2017-10-20 09:10:53,626 DEBUG
> [c.c.v.UserVmManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > > Successfully
>     >     > > > > > > allocated DB entry for VM[User|i-2-23-VM]
>     >     > > > > > > 2017-10-20 09:10:53,653 INFO
> [o.a.c.f.j.i.AsyncJobMonitor]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:0c9b9886) Add
>     >     > > > > job-181
>     >     > > > > > > into job monitoring
>     >     > > > > > > 2017-10-20 09:10:53,658 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > submit
>     >     > > > > async
>     >     > > > > > > job-181, details: AsyncJobVO {id:181, userId: 2,
>     > accountId: 2,
>     >     > > > > > > instanceType: VirtualMachine, instanceId: 23, cmd:
>     >     > > > > > > org.apache.cloudstack.api.command.admin.vm.
>     > DeployVMCmdByAdmin,
>     >     > > > > cmdInfo:
>     >     > > > > > > {"keyboard":"us","httpmethod":
>     > "GET","templateid":"244d6029-
>     >     > > > > > > 0f9a-4479-b11b-09e4fe6c40e1","
> securitygroupids":"1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe","
> ctxAccountId":"2","uuid":"
>     >     > > > > > > df7da1f1-4f17-4e98-9950-
> 84f30de1fcb2","cmdEventType":"
>     >     > > > > > > VM.CREATE","diskofferingid":"
> f39fb02b-4fc0-4f2e-9e4e-
>     >     > > > > > > dd54f7fc9508","size":"50","
> serviceofferingid":"e0ef2da9-
>     >     > > > > > > a489-4c42-ab6d-c49fd41b0fd2","
>     > response":"json","ctxUserId":"
>     >     > > > > > > 2","hypervisor":"KVM","
> displayname":"mtl-dev04-app02"
>     >     > > > > > > ,"name":"mtl-dev04-app02","
> zoneid":"b3d5b8bb-2dea-478b-
>     >     > > > > > > 8077-fc813b90d4c6","ctxStartEventId":"277","id":"
>     >     > > > > > > 23","ctxDetails":"{\"interface
>     >     > > > > > > com.cloud.offering.DiskOffering\":\"f39fb02b-
>     >     > > > > > > 4fc0-4f2e-9e4e-dd54f7fc9508\",\"interface
>     >     > > > > > > com.cloud.network.security.
> SecurityGroup\":\"1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe\",\"interface
>     >     > > > > > > com.cloud.vm.VirtualMachine\":
> \"df7da1f1-4f17-4e98-9950-
>     >     > > > > > > 84f30de1fcb2\",\"interface
>     >     > > > > > > com.cloud.dc.DataCenter\":\"b3d5b8bb-2dea-478b-8077-
>     >     > > > > > > fc813b90d4c6\",\"interface
>     >     > > > > > > com.cloud.offering.ServiceOffering\":\"e0ef2da9-
>     >     > > > > > > a489-4c42-ab6d-c49fd41b0fd2\",\"interface
>     >     > > > > > > com.cloud.template.VirtualMachineTemplate\":\"
>     >     > > > 244d6029-0f9a-4479-b11b-
>     >     > > > > > > 09e4fe6c40e1\"}","_":"1508490653550"},
>     >     > > > > > > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0,
>     > resultCode:
>     >     > > 0,
>     >     > > > > > > result: null, initMsid: 264216221068220,
> completeMsid:
>     > null,
>     >     > > > > lastUpdated:
>     >     > > > > > > null, lastPolled: null, created: null}
>     >     > > > > > > 2017-10-20 09:10:53,659 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0)
>     >     > > > Executing
>     >     > > > > > > AsyncJobVO {id:181, userId: 2, accountId: 2,
> instanceType:
>     >     > > > > > VirtualMachine,
>     >     > > > > > > instanceId: 23, cmd:
>     >     > > > > > > org.apache.cloudstack.api.command.admin.vm.
>     > DeployVMCmdByAdmin,
>     >     > > > > cmdInfo:
>     >     > > > > > > {"keyboard":"us","httpmethod":
>     > "GET","templateid":"244d6029-
>     >     > > > > > > 0f9a-4479-b11b-09e4fe6c40e1","
> securitygroupids":"1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe","
> ctxAccountId":"2","uuid":"
>     >     > > > > > > df7da1f1-4f17-4e98-9950-
> 84f30de1fcb2","cmdEventType":"
>     >     > > > > > > VM.CREATE","diskofferingid":"
> f39fb02b-4fc0-4f2e-9e4e-
>     >     > > > > > > dd54f7fc9508","size":"50","
> serviceofferingid":"e0ef2da9-
>     >     > > > > > > a489-4c42-ab6d-c49fd41b0fd2","
>     > response":"json","ctxUserId":"
>     >     > > > > > > 2","hypervisor":"KVM","
> displayname":"mtl-dev04-app02"
>     >     > > > > > > ,"name":"mtl-dev04-app02","
> zoneid":"b3d5b8bb-2dea-478b-
>     >     > > > > > > 8077-fc813b90d4c6","ctxStartEventId":"277","id":"
>     >     > > > > > > 23","ctxDetails":"{\"interface
>     >     > > > > > > com.cloud.offering.DiskOffering\":\"f39fb02b-
>     >     > > > > > > 4fc0-4f2e-9e4e-dd54f7fc9508\",\"interface
>     >     > > > > > > com.cloud.network.security.
> SecurityGroup\":\"1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe\",\"interface
>     >     > > > > > > com.cloud.vm.VirtualMachine\":
> \"df7da1f1-4f17-4e98-9950-
>     >     > > > > > > 84f30de1fcb2\",\"interface
>     >     > > > > > > com.cloud.dc.DataCenter\":\"b3d5b8bb-2dea-478b-8077-
>     >     > > > > > > fc813b90d4c6\",\"interface
>     >     > > > > > > com.cloud.offering.ServiceOffering\":\"e0ef2da9-
>     >     > > > > > > a489-4c42-ab6d-c49fd41b0fd2\",\"interface
>     >     > > > > > > com.cloud.template.VirtualMachineTemplate\":\"
>     >     > > > 244d6029-0f9a-4479-b11b-
>     >     > > > > > > 09e4fe6c40e1\"}","_":"1508490653550"},
>     >     > > > > > > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0,
>     > resultCode:
>     >     > > 0,
>     >     > > > > > > result: null, initMsid: 264216221068220,
> completeMsid:
>     > null,
>     >     > > > > lastUpdated:
>     >     > > > > > > null, lastPolled: null, created: null}
>     >     > > > > > > 2017-10-20 09:10:53,659 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-6:ctx-3911a4cc ctx-773820d9)
>     > (logid:41db964f)
>     >     > > > ===END===
>     >     > > > > > > 192.168.1.194 -- GET
>     >     > > > > > > command=deployVirtualMachine&
>     > response=json&zoneid=b3d5b8bb-
>     >     > > > > > > 2dea-478b-8077-fc813b90d4c6&
> templateid=244d6029-0f9a-4479-
>     >     > > > > > > b11b-09e4fe6c40e1&hypervisor=KVM&serviceofferingid=
>     >     > > > > > > e0ef2da9-a489-4c42-ab6d-c49fd41b0fd2&diskofferingid=
>     >     > > > > > > f39fb02b-4fc0-4f2e-9e4e-dd54f7fc9508&size=50&
>     >     > > > > securitygroupids=1586621a-
>     >     > > > > > > 8e8d-11e7-8f52-f04da2002bbe&
> displayname=mtl-dev04-app02&
>     >     > > > > > > name=mtl-dev04-app02&keyboard=us&_=1508490653550
>     >     > > > > > > 2017-10-20 09:10:53,688 DEBUG [c.c.d.
>     >     > > DeploymentPlanningManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > DeploymentPlanner allocation algorithm: null
>     >     > > > > > > 2017-10-20 09:10:53,688 DEBUG [c.c.d.
>     >     > > DeploymentPlanningManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Trying to allocate a host and storage pools from
> dc:1,
>     >     > > > > > > pod:null,cluster:null, requested cpu: 10000,
> requested ram:
>     >     > > > 8594128896
>     >     > > > > > > 2017-10-20 09:10:53,688 DEBUG [c.c.d.
>     >     > > DeploymentPlanningManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Is
>     >     > > > > > > ROOT volume READY (pool already allocated)?: No
>     >     > > > > > > 2017-10-20 09:10:53,691 DEBUG [c.c.d.
>     >     > > DeploymentPlanningManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Deploy avoids pods: [], clusters: [], hosts: []
>     >     > > > > > > 2017-10-20 09:10:53,691 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Searching all possible resources under this Zone: 1
>     >     > > > > > > 2017-10-20 09:10:53,692 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Listing clusters in order of aggregate capacity,
> that have
>     >     > (atleast
>     >     > > > one
>     >     > > > > > > host with) enough CPU and RAM capacity under this
> Zone: 1
>     >     > > > > > > 2017-10-20 09:10:53,693 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Removing from the clusterId list these clusters from
> avoid
>     > set:
>     >     > []
>     >     > > > > > > 2017-10-20 09:10:53,697 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Cannot allocate cluster list [1] for vm creation
> since
>     > their
>     >     > > > allocated
>     >     > > > > > > percentage crosses the disable capacity threshold
> defined
>     > at each
>     >     > > > > > cluster/
>     >     > > > > > > at global value for capacity Type : 1, skipping these
>     > clusters
>     >     > > > > > > 2017-10-20 09:10:53,697 DEBUG [c.c.d.FirstFitPlanner]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > No
>     >     > > > > > > clusters found after removing disabled clusters and
>     > clusters in
>     >     > > avoid
>     >     > > > > > list,
>     >     > > > > > > returning.
>     >     > > > > > > 2017-10-20 09:10:53,698 DEBUG
> [c.c.v.UserVmManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Destroying vm VM[User|i-2-23-VM] as it failed to
> create on
>     > Host
>     >     > > with
>     >     > > > > > > Id:null
>     >     > > > > > > 2017-10-20 09:10:53,701 DEBUG
> [c.c.c.CapacityManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > VM
>     >     > > > > > > state transitted from :Stopped to Error with event:
>     >     > > > > > > OperationFailedToErrorvm's original host id: null
> new host
>     > id:
>     >     > null
>     >     > > > > host
>     >     > > > > > id
>     >     > > > > > > before state transition: null
>     >     > > > > > > 2017-10-20 09:10:53,712 DEBUG
> [c.c.u.AccountManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Access granted to Acct[15863393-8e8d-11e7-8f52-
>     >     > f04da2002bbe-admin]
>     >     > > > to
>     >     > > > > > > Domain:1/ by AffinityGroupAccessChecker
>     >     > > > > > > 2017-10-20 09:10:53,713 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-5:null) (logid:) SeqA 5-377873:
>     > Processing
>     >     > > Seq
>     >     > > > > > > 5-377873:  { Cmd , MgmtId: -1, via: 5, Ver: v1,
> Flags: 11,
>     >     > > > > > > [{"com.cloud.agent.api.
> ConsoleProxyLoadReportCommand"
>     >     > > > > > > :{"_proxyVmId":1,"_loadInfo":"{\n
>     >     > > > > > > \"connections\": []\n}","wait":0}}] }
>     >     > > > > > > 2017-10-20 09:10:53,714 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-5:null) (logid:) SeqA 5-377873:
>     > Sending Seq
>     >     > > > > > > 5-377873:  { Ans: , MgmtId: 264216221068220, via: 5,
> Ver:
>     > v1,
>     >     > > Flags:
>     >     > > > > > > 100010,
>     >     > > > > > > [{"com.cloud.agent.api.
> AgentControlAnswer":{"result":
>     >     > > > true,"wait":0}}]
>     >     > > > > }
>     >     > > > > > > 2017-10-20 09:10:53,715 WARN  [o.a.c.alerts]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > alertType:: 8 // dataCenterId:: 1 // podId:: null //
>     > clusterId::
>     >     > > null
>     >     > > > > //
>     >     > > > > > > message:: Failed to deploy Vm with Id: 23, on Host
> with
>     > Id: null
>     >     > > > > > > 2017-10-20 09:10:53,726 INFO  [o.a.c.a.c.a.v.
>     > DeployVMCmdByAdmin]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > com.cloud.exception.InsufficientServerCapacityExce
> ption:
>     > Unable
>     >     > to
>     >     > > > > > create
>     >     > > > > > > a
>     >     > > > > > > deployment for VM[User|i-2-23-VM]Scope=interface
>     >     > > > > > com.cloud.dc.DataCenter;
>     >     > > > > > > id=1
>     >     > > > > > > 2017-10-20 09:10:53,726 INFO  [o.a.c.a.c.a.v.
>     > DeployVMCmdByAdmin]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181
> ctx-91a5717c)
>     >     > > > > (logid:25e762a0)
>     >     > > > > > > Unable to create a deployment for VM[User|i-2-23-VM]
>     >     > > > > > > com.cloud.exception.InsufficientServerCapacityExce
> ption:
>     > Unable
>     >     > to
>     >     > > > > > create
>     >     > > > > > > a
>     >     > > > > > > deployment for VM[User|i-2-23-VM]Scope=interface
>     >     > > > > > com.cloud.dc.DataCenter;
>     >     > > > > > > id=1
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.engine.cloud.entity.api.
>     >     > VMEntityManagerImpl.
>     >     > > > > > > reserveVirtualMachine(VMEntityManagerImpl.java:215)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.engine.cloud.entity.api.
>     >     > > > > VirtualMachineEntityImpl.
>     >     > > > > > > reserve(VirtualMachineEntityImpl.java:200)
>     >     > > > > > > at
>     >     > > > > > > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >     > > > > > > UserVmManagerImpl.java:4158)
>     >     > > > > > > at
>     >     > > > > > > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >     > > > > > > UserVmManagerImpl.java:3738)
>     >     > > > > > > at
>     >     > > > > > > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
>     >     > > > > > > UserVmManagerImpl.java:3726)
>     >     > > > > > > at sun.reflect.NativeMethodAccessorImpl.
> invoke0(Native
>     > Method)
>     >     > > > > > > at
>     >     > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(
>     >     > > > > > NativeMethodAccessorImpl.java:
>     >     > > > > > > 62)
>     >     > > > > > > at
>     >     > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
>     >     > > > > > > DelegatingMethodAccessorImpl.java:43)
>     >     > > > > > > at java.lang.reflect.Method.invoke(Method.java:498)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.support.AopUtils.
>     >     > > > > invokeJoinpointUsingReflection
>     >     > > > > > > (AopUtils.java:333)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.framework.
>     > ReflectiveMethodInvocation.
>     >     > > > > > > invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.framework.
>     > ReflectiveMethodInvocation.
>     >     > > > proceed(
>     >     > > > > > > ReflectiveMethodInvocation.java:157)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.network.contrail.management.
>     >     > > > > > > EventUtils$EventInterceptor.
> invoke(EventUtils.java:107)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.framework.
>     > ReflectiveMethodInvocation.
>     >     > > > proceed(
>     >     > > > > > > ReflectiveMethodInvocation.java:168)
>     >     > > > > > > at
>     >     > > > > > > com.cloud.event.ActionEventInterceptor.invoke(
>     >     > > > > > ActionEventInterceptor.java:
>     >     > > > > > > 51)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.framework.
>     > ReflectiveMethodInvocation.
>     >     > > > proceed(
>     >     > > > > > > ReflectiveMethodInvocation.java:168)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.interceptor.
>     > ExposeInvocationInterceptor.
>     >     > > > > invoke(
>     >     > > > > > > ExposeInvocationInterceptor.java:92)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.framework.
>     > ReflectiveMethodInvocation.
>     >     > > > proceed(
>     >     > > > > > > ReflectiveMethodInvocation.java:179)
>     >     > > > > > > at
>     >     > > > > > > org.springframework.aop.
> framework.JdkDynamicAopProxy.
>     >     > > > > > > invoke(JdkDynamicAopProxy.java:213)
>     >     > > > > > > at com.sun.proxy.$Proxy207.
> startVirtualMachine(Unknown
>     > Source)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.api.command.admin.vm.
>     >     > > > DeployVMCmdByAdmin.execute(
>     >     > > > > > > DeployVMCmdByAdmin.java:50)
>     >     > > > > > > at com.cloud.api.ApiDispatcher.
>     > dispatch(ApiDispatcher.java:150)
>     >     > > > > > > at
>     >     > > > > > > com.cloud.api.ApiAsyncJobDispatcher.runJob(
>     >     > > > ApiAsyncJobDispatcher.java:
>     >     > > > > > 108)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.framework.jobs.impl.
>     > AsyncJobManagerImpl$5.
>     >     > > > > > > runInContext(AsyncJobManagerImpl.java:558)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.managed.context.
>     >     > > ManagedContextRunnable$1.run(
>     >     > > > > > > ManagedContextRunnable.java:49)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.managed.context.impl.
>     >     > > > > DefaultManagedContext$1.call(
>     >     > > > > > > DefaultManagedContext.java:56)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.managed.context.impl.
>     >     > DefaultManagedContext.
>     >     > > > > > > callWithContext(DefaultManagedContext.java:103)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.managed.context.impl.
>     >     > DefaultManagedContext.
>     >     > > > > > > runWithContext(DefaultManagedContext.java:53)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.managed.context.
>     >     > ManagedContextRunnable.run(
>     >     > > > > > > ManagedContextRunnable.java:46)
>     >     > > > > > > at
>     >     > > > > > > org.apache.cloudstack.framework.jobs.impl.
>     >     > > AsyncJobManagerImpl$5.run(
>     >     > > > > > > AsyncJobManagerImpl.java:506)
>     >     > > > > > > at java.util.concurrent.Executors$RunnableAdapter.
>     >     > > > > > call(Executors.java:511)
>     >     > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.
>     > java:266)
>     >     > > > > > > at
>     >     > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
>     >     > > > > > > ThreadPoolExecutor.java:1149)
>     >     > > > > > > at
>     >     > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
>     >     > > > > > > ThreadPoolExecutor.java:624)
>     >     > > > > > > at java.lang.Thread.run(Thread.java:748)
>     >     > > > > > > 2017-10-20 09:10:53,726 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0)
>     >     > > Complete
>     >     > > > > > async
>     >     > > > > > > job-181, jobStatus: FAILED, resultCode: 530, result:
>     >     > > > > > > org.apache.cloudstack.api.
> response.ExceptionResponse/
>     >     > > > > > null/{"uuidList":[],"
>     >     > > > > > > errorcode":533,"errortext":"Unable
>     >     > > > > > > to create a deployment for VM[User|i-2-23-VM]"}
>     >     > > > > > > 2017-10-20 09:10:53,727 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0)
>     >     > > Publish
>     >     > > > > > async
>     >     > > > > > > job-181 complete on message bus
>     >     > > > > > > 2017-10-20 09:10:53,727 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0) Wake
>     >     > > up
>     >     > > > > jobs
>     >     > > > > > > related to job-181
>     >     > > > > > > 2017-10-20 09:10:53,727 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0)
>     >     > Update
>     >     > > > db
>     >     > > > > > > status for job-181
>     >     > > > > > > 2017-10-20 09:10:53,728 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0) Wake
>     >     > > up
>     >     > > > > jobs
>     >     > > > > > > joined with job-181 and disjoin all subjobs created
> from
>     > job- 181
>     >     > > > > > > 2017-10-20 09:10:53,729 DEBUG [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0) Done
>     >     > > > > > executing
>     >     > > > > > > org.apache.cloudstack.api.command.admin.vm.
>     > DeployVMCmdByAdmin
>     >     > for
>     >     > > > > > job-181
>     >     > > > > > > 2017-10-20 09:10:53,729 INFO
> [o.a.c.f.j.i.AsyncJobMonitor]
>     >     > > > > > > (API-Job-Executor-40:ctx-a4cff151 job-181)
>     > (logid:25e762a0)
>     >     > Remove
>     >     > > > > > job-181
>     >     > > > > > > from job monitoring
>     >     > > > > > > 2017-10-20 09:10:55,239 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-befa869b)
> (logid:3bfc7ad9)
>     > Begin
>     >     > > > cleanup
>     >     > > > > > > expired async-jobs
>     >     > > > > > > 2017-10-20 09:10:55,242 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-befa869b)
> (logid:3bfc7ad9)
>     > End
>     >     > > cleanup
>     >     > > > > > > expired
>     >     > > > > > > async-jobs
>     >     > > > > > > 2017-10-20 09:10:55,339 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-cc2ebd5d)
> (logid:00c6f41b)
>     > Found 0
>     >     > > > routers
>     >     > > > > to
>     >     > > > > > > update status.
>     >     > > > > > > 2017-10-20 09:10:55,340 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-cc2ebd5d)
> (logid:00c6f41b)
>     > Found 0
>     >     > VPC
>     >     > > > > > networks
>     >     > > > > > > to update Redundant State.
>     >     > > > > > > 2017-10-20 09:10:55,341 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-cc2ebd5d)
> (logid:00c6f41b)
>     > Found 0
>     >     > > > networks
>     >     > > > > > to
>     >     > > > > > > update RvR status.
>     >     > > > > > > 2017-10-20 09:10:56,695 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-13:ctx-8a748fc4) (logid:3489934a)
>     > ===START===
>     >     > > > > > > 192.168.1.194
>     >     > > > > > > -- GET
>     >     > > > > > > command=queryAsyncJobResult&
> jobId=25e762a0-615e-4cc7-8da0-
>     >     > > > > > > 66a178e0e5f9&response=json&_=1508490656682
>     >     > > > > > > 2017-10-20 09:10:56,706 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-13:ctx-8a748fc4 ctx-7b9abd96)
>     > (logid:3489934a)
>     >     > > > ===END===
>     >     > > > > > > 192.168.1.194 -- GET
>     >     > > > > > > command=queryAsyncJobResult&
> jobId=25e762a0-615e-4cc7-8da0-
>     >     > > > > > > 66a178e0e5f9&response=json&_=1508490656682
>     >     > > > > > > 2017-10-20 09:10:56,711 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-8:ctx-53eff2d0) (logid:f9178c9c)
> ===START===
>     >     > > > > > 192.168.1.194
>     >     > > > > > > -- GET
>     >     > > > > > > command=listVirtualMachines&
> id=df7da1f1-4f17-4e98-9950-
>     >     > > > > > > 84f30de1fcb2&response=json&_=1508490656699
>     >     > > > > > > 2017-10-20 09:10:56,733 DEBUG [c.c.a.ApiServlet]
>     >     > > > > > > (catalina-exec-8:ctx-53eff2d0 ctx-20971edb)
>     > (logid:f9178c9c)
>     >     > > > ===END===
>     >     > > > > > > 192.168.1.194 -- GET
>     >     > > > > > > command=listVirtualMachines&
> id=df7da1f1-4f17-4e98-9950-
>     >     > > > > > > 84f30de1fcb2&response=json&_=1508490656699
>     >     > > > > > > 2017-10-20 09:10:58,694 DEBUG [c.c.s.StatsCollector]
>     >     > > > > > > (StatsCollector-6:ctx-bcafae07) (logid:afeb8a5c)
>     >     > VmStatsCollector
>     >     > > is
>     >     > > > > > > running...
>     >     > > > > > > 2017-10-20 09:10:58,714 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-15:null) (logid:) SeqA
> 5-377874:
>     > Processing
>     >     > > Seq
>     >     > > > > > > 5-377874:  { Cmd , MgmtId: -1, via: 5, Ver: v1,
> Flags: 11,
>     >     > > > > > > [{"com.cloud.agent.api.
> ConsoleProxyLoadReportCommand"
>     >     > > > > > > :{"_proxyVmId":1,"_loadInfo":"{\n
>     >     > > > > > > \"connections\": []\n}","wait":0}}] }
>     >     > > > > > > 2017-10-20 09:10:58,716 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-15:null) (logid:) SeqA
> 5-377874:
>     > Sending
>     >     > Seq
>     >     > > > > > > 5-377874:  { Ans: , MgmtId: 264216221068220, via: 5,
> Ver:
>     > v1,
>     >     > > Flags:
>     >     > > > > > > 100010,
>     >     > > > > > > [{"com.cloud.agent.api.
> AgentControlAnswer":{"result":
>     >     > > > true,"wait":0}}]
>     >     > > > > }
>     >     > > > > > > 2017-10-20 09:10:58,761 DEBUG [c.c.s.StatsCollector]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27)
>     >     > StorageCollector
>     >     > > is
>     >     > > > > > > running...
>     >     > > > > > > 2017-10-20 09:10:58,766 DEBUG
>     > [c.c.h.o.r.Ovm3HypervisorGuru]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27)
>     >     > > > > > getCommandHostDelegation:
>     >     > > > > > > class com.cloud.agent.api.GetStorageStatsCommand
>     >     > > > > > > 2017-10-20 09:10:58,766 DEBUG [c.c.h.XenServerGuru]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27)
>     >     > > > > > getCommandHostDelegation:
>     >     > > > > > > class com.cloud.agent.api.GetStorageStatsCommand
>     >     > > > > > > 2017-10-20 09:10:58,792 DEBUG [c.c.a.t.Request]
>     >     > > > > > > (StatsCollector-6:ctx-bcafae07) (logid:afeb8a5c) Seq
>     >     > > > > > > 3-6621135877164909219:
>     >     > > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
>     >     > 3(mtl1-apphst03),
>     >     > > > > Ver:
>     >     > > > > > > v1, Flags: 10, { GetVmStatsAnswer } }
>     >     > > > > > > 2017-10-20 09:10:58,813 DEBUG [c.c.a.t.Request]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27) Seq
>     >     > > > > > > 4-1596244592926179066:
>     >     > > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
>     > 4(s-2-VM), Ver:
>     >     > > v1,
>     >     > > > > > > Flags: 10, { GetStorageStatsAnswer } }
>     >     > > > > > > 2017-10-20 09:10:58,814 DEBUG
>     > [c.c.h.o.r.Ovm3HypervisorGuru]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27)
>     >     > > > > > getCommandHostDelegation:
>     >     > > > > > > class com.cloud.agent.api.GetStorageStatsCommand
>     >     > > > > > > 2017-10-20 09:10:58,814 DEBUG [c.c.h.XenServerGuru]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27)
>     >     > > > > > getCommandHostDelegation:
>     >     > > > > > > class com.cloud.agent.api.GetStorageStatsCommand
>     >     > > > > > > 2017-10-20 09:10:58,881 DEBUG [c.c.a.t.Request]
>     >     > > > > > > (StatsCollector-1:ctx-fe08de8f) (logid:a41fef27) Seq
>     >     > > > > > > 3-6621135877164909220:
>     >     > > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
>     >     > 3(mtl1-apphst03),
>     >     > > > > Ver:
>     >     > > > > > > v1, Flags: 10, { GetStorageStatsAnswer } }
>     >     > > > > > > 2017-10-20 09:11:05,239 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-b43d5944)
> (logid:dbf81eb8)
>     > Begin
>     >     > > > cleanup
>     >     > > > > > > expired async-jobs
>     >     > > > > > > 2017-10-20 09:11:05,242 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-b43d5944)
> (logid:dbf81eb8)
>     > End
>     >     > > cleanup
>     >     > > > > > > expired
>     >     > > > > > > async-jobs
>     >     > > > > > > 2017-10-20 09:11:08,714 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-1:null) (logid:) SeqA 5-377875:
>     > Processing
>     >     > > Seq
>     >     > > > > > > 5-377875:  { Cmd , MgmtId: -1, via: 5, Ver: v1,
> Flags: 11,
>     >     > > > > > > [{"com.cloud.agent.api.
> ConsoleProxyLoadReportCommand"
>     >     > > > > > > :{"_proxyVmId":1,"_loadInfo":"{\n
>     >     > > > > > > \"connections\": []\n}","wait":0}}] }
>     >     > > > > > > 2017-10-20 09:11:08,716 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-1:null) (logid:) SeqA 5-377875:
>     > Sending Seq
>     >     > > > > > > 5-377875:  { Ans: , MgmtId: 264216221068220, via: 5,
> Ver:
>     > v1,
>     >     > > Flags:
>     >     > > > > > > 100010,
>     >     > > > > > > [{"com.cloud.agent.api.
> AgentControlAnswer":{"result":
>     >     > > > true,"wait":0}}]
>     >     > > > > }
>     >     > > > > > > 2017-10-20 09:11:10,978 DEBUG [c.c.c.
>     > ConsoleProxyManagerImpl]
>     >     > > > > > > (consoleproxy-1:ctx-db1623b9) (logid:5ce90094) Zone
> 1 is
>     > ready to
>     >     > > > > launch
>     >     > > > > > > console proxy
>     >     > > > > > > 2017-10-20 09:11:11,058 DEBUG [o.a.c.s.
>     >     > > SecondaryStorageManagerImpl]
>     >     > > > > > > (secstorage-1:ctx-5c3bea1b) (logid:dab66628) Zone 1
> is
>     > ready to
>     >     > > > launch
>     >     > > > > > > secondary storage VM
>     >     > > > > > > 2017-10-20 09:11:12,160 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-9:null) (logid:) Ping from
> 5(v-1-VM)
>     >     > > > > > > 2017-10-20 09:11:15,239 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-60c073b3)
> (logid:cd140ec0)
>     > Begin
>     >     > > > cleanup
>     >     > > > > > > expired async-jobs
>     >     > > > > > > 2017-10-20 09:11:15,242 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-60c073b3)
> (logid:cd140ec0)
>     > End
>     >     > > cleanup
>     >     > > > > > > expired
>     >     > > > > > > async-jobs
>     >     > > > > > > 2017-10-20 09:11:15,766 DEBUG [c.c.s.StatsCollector]
>     >     > > > > > > (StatsCollector-2:ctx-5d5c1cb9) (logid:d0b73976)
>     >     > > HostStatsCollector
>     >     > > > is
>     >     > > > > > > running...
>     >     > > > > > > 2017-10-20 09:11:15,814 DEBUG [c.c.a.t.Request]
>     >     > > > > > > (StatsCollector-2:ctx-5d5c1cb9) (logid:d0b73976) Seq
>     >     > > > > > > 3-6621135877164909221:
>     >     > > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
>     >     > 3(mtl1-apphst03),
>     >     > > > > Ver:
>     >     > > > > > > v1, Flags: 10, { GetHostStatsAnswer } }
>     >     > > > > > > 2017-10-20 09:11:16,065 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-2:null) (logid:) Ping from
> 4(s-2-VM)
>     >     > > > > > > 2017-10-20 09:11:18,715 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-7:null) (logid:) SeqA 5-377877:
>     > Processing
>     >     > > Seq
>     >     > > > > > > 5-377877:  { Cmd , MgmtId: -1, via: 5, Ver: v1,
> Flags: 11,
>     >     > > > > > > [{"com.cloud.agent.api.
> ConsoleProxyLoadReportCommand"
>     >     > > > > > > :{"_proxyVmId":1,"_loadInfo":"{\n
>     >     > > > > > > \"connections\": []\n}","wait":0}}] }
>     >     > > > > > > 2017-10-20 09:11:18,717 DEBUG
> [c.c.a.m.AgentManagerImpl]
>     >     > > > > > > (AgentManager-Handler-7:null) (logid:) SeqA 5-377877:
>     > Sending Seq
>     >     > > > > > > 5-377877:  { Ans: , MgmtId: 264216221068220, via: 5,
> Ver:
>     > v1,
>     >     > > Flags:
>     >     > > > > > > 100010,
>     >     > > > > > > [{"com.cloud.agent.api.
> AgentControlAnswer":{"result":
>     >     > > > true,"wait":0}}]
>     >     > > > > }
>     >     > > > > > > 2017-10-20 09:11:25,240 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-df0997a3)
> (logid:d8ea9924)
>     > Begin
>     >     > > > cleanup
>     >     > > > > > > expired async-jobs
>     >     > > > > > > 2017-10-20 09:11:25,243 INFO  [o.a.c.f.j.i.
>     > AsyncJobManagerImpl]
>     >     > > > > > > (AsyncJobMgr-Heartbeat-1:ctx-df0997a3)
> (logid:d8ea9924)
>     > End
>     >     > > cleanup
>     >     > > > > > > expired
>     >     > > > > > > async-jobs
>     >     > > > > > > 2017-10-20 09:11:25,268 DEBUG [c.c.n.
>     >     > > ExternalDeviceUsageManagerImpl]
>     >     > > > > > > (ExternalNetworkMonitor-1:ctx-148fd210)
> (logid:3a3d51da)
>     >     > External
>     >     > > > > > devices
>     >     > > > > > > are not used. Skipping external device usage
> collection
>     >     > > > > > > 2017-10-20 09:11:25,310 INFO  [c.c.h.
>     >     > HighAvailabilityManagerImpl]
>     >     > > > > > > (HA-5:ctx-f691d0f0) (logid:45cb53d0) checking health
> of
>     > usage
>     >     > > server
>     >     > > > > > > 2017-10-20 09:11:25,311 DEBUG [c.c.h.
>     >     > HighAvailabilityManagerImpl]
>     >     > > > > > > (HA-5:ctx-f691d0f0) (logid:45cb53d0) usage server
> running?
>     > false,
>     >     > > > > > > heartbeat: null
>     >     > > > > > > 2017-10-20 09:11:25,312 WARN  [o.a.c.alerts]
>     > (HA-5:ctx-f691d0f0)
>     >     > > > > > > (logid:45cb53d0)  alertType:: 13 // dataCenterId:: 0
> //
>     > podId:: 0
>     >     > > //
>     >     > > > > > > clusterId:: null // message:: No usage server process
>     > running
>     >     > > > > > > 2017-10-20 09:11:25,313 DEBUG
> [c.c.a.AlertManagerImpl]
>     >     > > > > > (HA-5:ctx-f691d0f0)
>     >     > > > > > > (logid:45cb53d0) Have already sent: 1 emails for
> alert
>     > type '13'
>     >     > --
>     >     > > > > > > skipping send email
>     >     > > > > > > 2017-10-20 09:11:25,338 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterMonitor-1:ctx-e46df5fd) (logid:7cdfdeb5)
> Found 0
>     > running
>     >     > > > > routers.
>     >     > > > > > > 2017-10-20 09:11:25,339 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-04de4525)
> (logid:a35da4f6)
>     > Found 0
>     >     > > > routers
>     >     > > > > to
>     >     > > > > > > update status.
>     >     > > > > > > 2017-10-20 09:11:25,340 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-04de4525)
> (logid:a35da4f6)
>     > Found 0
>     >     > VPC
>     >     > > > > > networks
>     >     > > > > > > to update Redundant State.
>     >     > > > > > > 2017-10-20 09:11:25,340 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-04de4525)
> (logid:a35da4f6)
>     > Found 0
>     >     > > > networks
>     >     > > > > > to
>     >     > > > > > > update RvR status.
>     >     > > > > > > 2017-10-20 09:11:25,349 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-47e38698)
> (logid:d39e2330)
>     > Found 1
>     >     > > > running
>     >     > > > > > > routers.
>     >     > > > > > > 2017-10-20 09:11:25,351 DEBUG [c.c.n.r.
>     >     > > > VirtualNetworkApplianceManager
>     >     > > > > > Impl]
>     >     > > > > > > (RouterStatusMonitor-1:ctx-47e38698)
> (logid:d39e2330) IP
>     > of
>     >     > > control
>     >     > > > > NIC
>     >     > > > > > > 169.254.0.113
>     >     > > > > > >
>     >     > > > > >
>     >     > > > > >
>     >     > > > > >
>     >     > > > > > --
>     >     > > > > > With best regards, Ivan Kudryavtsev
>     >     > > > > > Bitworks Software, Ltd.
>     >     > > > > > Cell: +7-923-414-1515
>     >     > > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
>     >     > > > > >
>     >     > > > >
>     >     > > >
>     >     > > >
>     >     > > >
>     >     > > > --
>     >     > > > With best regards, Ivan Kudryavtsev
>     >     > > > Bitworks Software, Ltd.
>     >     > > > Cell: +7-923-414-1515
>     >     > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
>     >     > > >
>     >     > >
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     > With best regards, Ivan Kudryavtsev
>     >     > Bitworks Software, Ltd.
>     >     > Cell: +7-923-414-1515
>     >     > WWW: http://bitworks.software/ <http://bw-sw.com/>
>     >     >
>     >
>     >
>     >
>     > Dag.Sonstebo@shapeblue.com
>     > www.shapeblue.com
>     > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>     > @shapeblue
>     >
>     >
>     >
>     >
>
>
>
> Dag.Sonstebo@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message