cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evan Miller <Evan.Mil...@citrix.com>
Subject RE: Unable to add an instance - speed=500Mhz, Ram=512 (host is in avoid set)
Date Tue, 05 Jun 2012 19:52:00 GMT
Attached is the job-18 log snippet.
Sorry for not including this attachment with the previous email.

Regards,
Evan

-----Original Message-----
From: Evan Miller [mailto:Evan.Miller@citrix.com]
Sent: Tuesday, June 05, 2012 12:48 PM
To: cloudstack-users@incubator.apache.org
Subject: RE: Unable to add an instance - speed=500Mhz, Ram=512 (host is in avoid set)


-----Original Message-----
From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
Sent: Tuesday, June 05, 2012 12:08 PM
To: cloudstack-users@incubator.apache.org
Subject: Re: Unable to add an instance - speed=500Mhz, Ram=512 (host is in avoid set)

On 6/5/12 11:49 AM, "Evan Miller" <Evan.Miller@citrix.com> wrote:

>RECOMMENDATION:
>Need to improve the 3.0.0 - 3.0.2 Basic Installation Guide.
>I don't have huge ranges of ip addresses available. So, it would be
>extremely beneficial to have some idea, in advance, as to how ip
>addresses will be consumed by a basic cloud setup. For example, I was
>surprised to learn that in my basic setup, one CloudStack Management
>Server (also serving primary and secondary filesystems) and one
>XenServer using one connected nic, the following ip addresses were
>required even before adding a single VM instance:
>
>  CSMS - 1 ip (management)  [not surprising]  XenServer - 1 ip
> (management) [not surprising]  s-1-VM - 3 ips (2 - management, 1 -
> guest) [surprising]  v-2-VM - 2 ips (1 - management, 1 - guest)
> [surprising]
>
>So, for each VM instance, is only one guest ip address required? Or is
>more required?

Only one guest IP.

>> After adding the guest IP to the pod, still not able to add a VM
>> instance to the cloud. Goth this popup error window:

>>   Unable to create a deployment for VM[User|i-2-9-VM]

>> Seeing insufficient capacity for job-18 in management-server.log
>> Attaching the log file snippet. It is hard for me to determine the
>> precise insufficiency. Please let me know.

>> Also, the dashboard shows that the fourth guest ip address is still
>> taken. When will CSMS give back the ip address of the VM instance
>> that failed? Logging out and redisplaying the dashboard from the
>> browser still shows the fourth direct ip is taken. Destroying all
>> failed instances didn't help either. Clinging to that ip address
>> seems like a bug to me.

>> Also, how long does expunging take after I destroy an instance?
>> That is, when does the instance finally become expunged?
>> I see instances expunging for about 24 hours now.

>> Regards,
>> Evan

>
>Also, I, actually, don't see in XenCenter 6.0 what device is using the
>end (last) guest ip address that I defined. However, it is apparently
>used by something. What is using the last
>(end) guest ip address? The dashboard says the 3 direct ip addresses
>are used, but I can't identify what is using the third (last or end)
>guest ip address.



You can check cloud.nics table (select instance_id where ip4_address=<ip> and network_id=<id
of the guest network>

>
>
>Regards,
>Evan
>
>-----Original Message-----
>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>Sent: Monday, June 04, 2012 9:14 PM
>To: cloudstack-users@incubator.apache.org
>Subject: Re: Unable to add an instance - speed=500Mhz, Ram=512 (host is
>in avoid set)
>
>On 6/4/12 8:07 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>
>>Attached is the proper grep of job-14.
>>
>>In this small size instance scenario failure, is the XenServer the
>>system that is saying 'Hey 500Mbytes isn't enough RAM'?
>>
>>I am trying to understand exactly which system is saying what in
>>particular is insufficient.
>
>
>
>Here is the reason for the failure: Nic reservation failed due to
>Insufficient capacity exception (DirectPodBasedNetworkGuru.getIp:158):
>
>
>2012-06-04 18:10:02,577 DEBUG [db.Transaction.Transaction]
>(Job-Executor-1:job-14) Rolling back the transaction: Time = 2 Name =
>-AsyncJobManagerImpl$1.run:393-Executors$RunnableAdapter.call:471-Futur
>eTa
>s
>k$Sync.innerRun:334-FutureTask.run:166-ThreadPoolExecutor.runWorker:111
>0-T
>h
>readPoolExecutor$Worker.run:603-Thread.run:679; called by
>-Transaction.rollback:837-Transaction.removeUpTo:780-Transaction.close:
>599
>-
>DatabaseCallback.interceptComplete:67-DatabaseCallback.intercept:32-Net
>wor
>k
>ManagerImpl.assignPublicIpAddress:355-DirectPodBasedNetworkGuru.getIp:1
>58-
>D
>irectPodBasedNetworkGuru.reserve:146-DatabaseCallback.intercept:30-Netw
>ork
>M
>anagerImpl.prepare:1859-VirtualMachineManagerImpl.advanceStart:741-Virt
>ual
>M
>achineManagerImpl.start:461
>2012-06-04 18:10:02,578 INFO  [cloud.vm.VirtualMachineManagerImpl]
>(Job-Executor-1:job-14) Insufficient capacity
>
>
>Looks like the Guest network doesn't have free guest IPs in the POD1
>targeted for deployment. To add the additional Ips, do this:
>
>* select Zone
>* Select Guest Network on the network diagram
>* Add guest IP range to the POD.
>
>
>-Alena.
>
>
>
>
>>
>>
>>Regards,
>>Evan
>>
>>-----Original Message-----
>>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>>Sent: Monday, June 04, 2012 7:41 PM
>>To: cloudstack-users@incubator.apache.org
>>Subject: Re: Unable to add an instance - speed=500Mhz, Ram=512 (host
>>is in avoid set)
>>
>>The setup lacks the resources. See more detailed information inline.
>>Btw, to capture all logs related to vm deployment, grep by job-id
>>should be done (in this case "grep job-14 management-server.log").
>>
>>-Alena.
>>
>>
>>On 6/4/12 7:27 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>
>>>Trying to setup a single, small VM to my cloud.
>>>My cloud, so far, consists of:
>>>
>>>CloudStack Management Server:
>>> *   OS = Centos 6.2 x86 64-bit
>>> *   CPU = Intel 3050, 2.13 GHz
>>> *   RAM = 2 Gbytes
>>>
>>>XenServer:
>>> *   OS = Provided by XenServer 6.0.2 CD
>>> *   Hardware = IBM 3250
>>> *   CPU = Intel Xeon Dual Core, Single Physical CPU@ 1.86 GHz
>>>
>>>Template:
>>>* CentOS 5.6(64-bit) no GUI (XenServer)
>>>
>>>Per the management log below, is XenServer refusing to setup the
>>>first VM?
>>>
>>>
>>>2012-06-04 18:10:02,907 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) DeploymentPlanner allocation algorithm:
>>>random
>>>2012-06-04 18:10:02,907 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Trying to allocate a host and storage pools
>>>from dc:1, pod:null,cluster:null, requested cpu: 500, requested ram:
>>>536870912
>>>2012-06-04 18:10:02,907 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Is ROOT volume READY (pool already allocated)?:
>>>No
>>>2012-06-04 18:10:02,907 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Searching all possible resources under this
>>>Zone:
>>>1
>>>2012-06-04 18:10:02,909 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Listing clusters in order of aggregate
>>>capacity, that have (atleast one host with) enough CPU and RAM
>>>capacity under this
>>>Zone: 1
>>>2012-06-04 18:10:02,910 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) CPUOverprovisioningFactor considered: 1.0
>>>2012-06-04 18:10:02,918 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Checking resources in Cluster: 1 under Pod: 1
>>>2012-06-04 18:10:02,918 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Calling HostAllocators to find suitable hosts
>>>2012-06-04 18:10:02,918 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) Looking for hosts in
>>>dc:
>>>1  pod:1  cluster:1
>>>2012-06-04 18:10:02,920 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) FirstFitAllocator
>>>has
>>>1 hosts to check for allocation: [Host[-1-Routing]]
>>>2012-06-04 18:10:02,923 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) Found 1 hosts for
>>>allocation after prioritization: [Host[-1-Routing]]
>>>2012-06-04 18:10:02,923 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) Looking for
>>>speed=500Mhz, Ram=512
>>>2012-06-04 18:10:02,923 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) Host name:
>>>xenserver1-cs, hostId: 1 is in avoid set, skipping this and trying
>>>other available hosts
>>
>>
>>The host was put in avoid set some time before (the log fragment
>>you've sent doesn't capture it). The host can be put in avoid set when
>>it doesn't have enough resources for vm start (cpu/ram).
>>
>>
>>
>>>2012-06-04 18:10:02,923 DEBUG [allocator.impl.FirstFitAllocator]
>>>(Job-Executor-1:job-14 FirstFitRoutingAllocator) Host Allocator
>>>returning 0 suitable hosts
>>>2012-06-04 18:10:02,923 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) No suitable hosts found
>>>2012-06-04 18:10:02,923 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) No suitable hosts found under this Cluster: 1
>>>2012-06-04 18:10:02,923 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-1:job-14) Could not find suitable Deployment
>>>Destination for this VM under any clusters, returning.
>>>2012-06-04 18:10:03,022 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-1:job-14) VM state transitted from :Starting to Stopped
>>>with event: OperationFailedvm's original host id: null new host id:
>>>null host id before state transition: 1
>>>2012-06-04 18:10:03,027 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-1:job-14) Hosts's actual total CPU: 3732 and CPU after
>>>applying overprovisioning: 3732
>>>2012-06-04 18:10:03,028 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-1:job-14) release cpu from host: 1, old used:
>>>1500,reserved: 0, actual total: 3732, total with overprovisioning:
>>>3732; new used: 1000,reserved:0; movedfromreserved:
>>>false,moveToReserveredfalse
>>>2012-06-04 18:10:03,028 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-1:job-14) release mem from host: 1, old used:
>>>1879048192,reserved: 0, total: 3775173696; new used:
>>>1342177280,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>>2012-06-04 18:10:03,183 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-1:job-14) VM state transitted from :Stopped to Error
>>>with
>>>event: OperationFailedToErrorvm's original host id: null new host id:
>>>null host id before state transition: null
>>>2012-06-04 18:10:03,888 INFO  [api.commands.DeployVMCmd]
>>>(Job-Executor-1:job-14)
>>>com.cloud.exception.InsufficientServerCapacityException: Unable to
>>>create a deployment for VM[User|i-2-7-VM]Scope=interface
>>>com.cloud.dc.DataCenter; id=1
>>>2012-06-04 18:10:03,888 WARN  [cloud.api.ApiDispatcher]
>>>(Job-Executor-1:job-14) class com.cloud.api.ServerApiException :
>>>Unable to create a deployment for VM[User|i-2-7-VM]
>>>2012-06-04 18:10:03,891 DEBUG [cloud.async.AsyncJobManagerImpl]
>>>(Job-Executor-1:job-14) Complete async job-14, jobStatus: 2, resultCode:
>>>530, result:
>>>com.cloud.api.response.ExceptionResponse@4ebe0382<mailto:com.cloud.api.
>>>res
>>>ponse.ExceptionResponse@4ebe0382>
>>>2012-06-04 18:10:07,258 DEBUG [cloud.async.AsyncJobManagerImpl]
>>>(catalina-exec-7:null) Async job-14 completed
>>>
>>>
>>>
>>>I, also, tried to create a medium size VM. This, also, failed with a
>>>different error related to InsufficientServerCapacityException:
>>>
>>>
>>>
>>>2012-06-04 17:34:52,550 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) DeploymentPlanner allocation algorithm: random
>>>2012-06-04 17:34:52,550 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Trying to allocate a host and storage pools
>>>from dc:1, pod:null,cluster:null, requested cpu: 1000, requested ram:
>>>1073741824
>>>2012-06-04 17:34:52,550 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Is ROOT volume READY (pool already allocated)?:
>>>No
>>>2012-06-04 17:34:52,550 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Searching all possible resources under this
>>>Zone: 1
>>>2012-06-04 17:34:52,551 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Listing clusters in order of aggregate
>>>capacity, that have (atleast one host with) enough CPU and RAM
>>>capacity under this
>>>Zone: 1
>>>2012-06-04 17:34:52,552 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) CPUOverprovisioningFactor considered: 1.0
>>>2012-06-04 17:34:52,558 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Cannot allocate cluster 1 for vm creation
>>>since its allocated percentage: 0.9243709585329766 will cross the
>>>disable capacity threshold: 0.85 for capacity Type : 0, skipping this
>>>cluster
>>
>>
>>CapacityType: 0 means RAM. When we analyze if host is capable for vm
>>deployment, we always make sure the total RAM (including the RAM we
>>are allocating for the vm), doesn't exceed RAM capacity threshold
>>(global config variable cluster.memory.allocated.capacity.disablethreshold).
>>
>>
>>
>>>2012-06-04 17:34:52,558 DEBUG [cloud.deploy.FirstFitPlanner]
>>>(Job-Executor-9:job-9) Could not find suitable Deployment Destination
>>>for this VM under any clusters, returning.
>>>2012-06-04 17:34:52,690 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-9:job-9) VM state transitted from :Starting to Stopped
>>>with
>>>event: OperationFailedvm's original host id: null new host id: null
>>>host id before state transition: 1
>>>2012-06-04 17:34:52,694 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-9:job-9) Hosts's actual total CPU: 3732 and CPU after
>>>applying overprovisioning: 3732
>>>2012-06-04 17:34:52,694 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-9:job-9) release cpu from host: 1, old used:
>>>2000,reserved:
>>>0, actual total: 3732, total with overprovisioning: 3732; new used:
>>>1000,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>>2012-06-04 17:34:52,694 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-9:job-9) release mem from host: 1, old used:
>>>2415919104,reserved: 0, total: 3775173696; new used:
>>>1342177280,reserved:0; movedfromreserved: false,moveToReserveredfalse
>>>2012-06-04 17:34:52,842 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>(Job-Executor-9:job-9) VM state transitted from :Stopped to Error
>>>with
>>>event: OperationFailedToErrorvm's original host id: null new host id:
>>>null host id before state transition: null
>>>2012-06-04 17:34:53,463 INFO  [api.commands.DeployVMCmd]
>>>(Job-Executor-9:job-9)
>>>com.cloud.exception.InsufficientServerCapacityException: Unable to
>>>create a deployment for VM[User|i-2-5-VM]Scope=interface
>>>com.cloud.dc.DataCenter; id=1
>>>2012-06-04 17:34:53,464 WARN  [cloud.api.ApiDispatcher]
>>>(Job-Executor-9:job-9) class com.cloud.api.ServerApiException :
>>>Unable to create a deployment for VM[User|i-2-5-VM]
>>>2012-06-04 17:34:53,464 DEBUG [cloud.async.AsyncJobManagerImpl]
>>>(Job-Executor-9:job-9) Complete async job-9, jobStatus: 2, resultCode:
>>>530, result:
>>>com.cloud.api.response.ExceptionResponse@7f378496<mailto:com.cloud.api.
>>>res
>>>ponse.ExceptionResponse@7f378496>
>>>2012-06-04 17:34:56,985 DEBUG [cloud.async.AsyncJobManagerImpl]
>>>(catalina-exec-17:null) Async job-9 completed
>>>
>>>How to overcome these errors?
>>
>>
>>The solution will be - create the service offering with small amount
>>of RAM/CPU (depends of how many vms you are planning to run on your host).
>>Or add more hosts to the Xen cluster.
>>
>>>
>>>Regards,
>>>
>>>Evan Miller
>>>
>>>
>>
>>
>>
>
>
>



Mime
View raw message