cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evan Miller <Evan.Mil...@citrix.com>
Subject RE: Problem: Unable to create a deployment for VM[User|i-2-8-VM]
Date Tue, 17 Jul 2012 18:50:46 GMT
I used the Basic Network Setup Wizard to setup my cloud.
So, those three "system" VMs were automatically created for
me. That's the context for my use of the term "automatic".

Regards,
Evan


-----Original Message-----
From: Evan Miller
Sent: Tuesday, July 17, 2012 11:47 AM
To: 'cloudstack-users@incubator.apache.org'
Subject: RE: Problem: Unable to create a deployment for VM[User|i-2-8-VM]

These guys were running:
s-1-VM
v-2-VM
r-4-VM
but were created by CSMS behind the scenes.

What I meant was there were no VMs that I explicitly created using the GUI by "adding an instance"
that were still running.

In my opinion, if CSMS automatically creates VMs (say those three system VMs), then CSMS should
automatically manage them as well. Why during deletion of a different device (for example
a host) should I suddenly have to think about other systems that CSMS automatically created
previously?

Regards,
Evan



-----Original Message-----
From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
Sent: Tuesday, July 17, 2012 11:44 AM
To: cloudstack-users@incubator.apache.org
Subject: Re: Problem: Unable to create a deployment for VM[User|i-2-8-VM]

On 7/17/12 11:20 AM, "Evan Miller" <Evan.Miller@citrix.com> wrote:

>host id=1 was always removed using the API.
>
>I don't believe any non-system instances were ever running when
>deleteHost was executed.
>
>No error was ever observed after running deleteHost API.
>
>Regards,
>Evan



But in your previous email you've mentioned that there were vms running on the host id=1:

mysql> Select * from vm_instance where state in ('Running', 'Starting',
'Stopping') AND host_id in (select id from host where removed is not
null);
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+-
----+----------------+-------------+---------------------+--------------
----+----------------+-------------+---------------------+---
---+--------+----------------+---------+--------------+----------+------
---+--------+----------------+---------+--------------+----------+---
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----+--------------------+--------------------+------------+-----------+--
-------------------+--------------------------------------+-------------
-------------------+--------------------------------------+---
-+
| id | name   | uuid                                 | instance_name |
state   | vm_template_id | guest_os_id | private_mac_address |
private_ip_address | pod_id | data_center_id | host_id | last_host_id | proxy_id | proxy_assign_time
| vnc_password
 | ha_enabled | limit_cpu_use | update_count | update_time         |
created             | removed | type               | vm_type            |
account_id | domain_id | service_offering_id | reservation_id
         | hypervisor_type |
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+-
----+----------------+-------------+---------------------+--------------
----+----------------+-------------+---------------------+---
---+--------+----------------+---------+--------------+----------+------
---+--------+----------------+---------+--------------+----------+---
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----+--------------------+--------------------+------------+-----------+--
-------------------+--------------------------------------+-------------
-------------------+--------------------------------------+---
-+
|  1 | s-1-VM | 4780c7ba-dd55-45a2-a2e5-ed2974c070b5 | s-1-VM        |
Running |              1 |          15 | 06:9e:ec:00:00:07   |
10.217.5.199       |      1 |              1 |       1 |            1 |
 NULL | NULL              | caqFANzAxnvRMnVMJeajvIY4x3W8NMjFUkj/RkQmhSU= |
         0 |             0 |            3 | 2012-07-14 00:02:27 |
2012-07-13 23:59:42 | NULL    | SecondaryStorageVm | SecondaryStorageVm |
        1 |         1 |                   6 |
e62a93bd-9bb0-4640-8a89-dc1049f300fa | XenServer       |
|  2 | v-2-VM | 79924f6e-66e2-429f-b38a-9d3bd3f6a575 | v-2-VM        |
Running |              1 |          15 | 06:31:80:00:00:06   |
10.217.5.198       |      1 |              1 |       1 |            1 |
 NULL | NULL              | cF8VnXqcjTMS8dUfURYlShHJhV55iUFKMyyEKA/SNhw= |
         0 |             0 |            3 | 2012-07-14 00:04:19 |
2012-07-13 23:59:42 | NULL    | ConsoleProxy       | ConsoleProxy       |
        1 |         1 |                   8 |
c1e4d053-95c6-4804-b73f-c38e7e4e887a | XenServer       |
|  4 | r-4-VM | 593d5b00-9c73-485f-b091-121b69171e65 | r-4-VM        |
Running |              1 |          15 | 0e:00:a9:fe:00:88   |
169.254.0.136      |      1 |              1 |       1 |            1 |
 NULL | NULL              | AZ69gd1Mp6IO4gt0AWAeAQawEXZtjh9mA7cQ6ZCDcJg= |
         1 |             0 |            3 | 2012-07-14 01:53:39 |
2012-07-14 01:52:48 | NULL    | DomainRouter       | DomainRouter       |
        1 |         1 |                   7 |
4f69471a-c0f0-48d2-a39e-3dda20b20c56 | XenServer       |
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+--
+----+--------+--------------------------------------+---------------+-
----+----------------+-------------+---------------------+--------------
----+----------------+-------------+---------------------+---
---+--------+----------------+---------+--------------+----------+------
---+--------+----------------+---------+--------------+----------+---
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
----------+----------------------------------------------+------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----------+--------------+---------------------+---------------------+-
-----+--------------------+--------------------+------------+-----------+--
-------------------+--------------------------------------+-------------
-------------------+--------------------------------------+---
-+
3 rows in set (0.00 sec)




I would advise to file a bug; and attach management server log file from the day where host
was removed + the day vm r-4-VM was started.

-Alena.

>
>
>-----Original Message-----
>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>Sent: Monday, July 16, 2012 7:25 PM
>To: cloudstack-users@incubator.apache.org
>Subject: Re: Problem: Unable to create a deployment for
>VM[User|i-2-8-VM]
>
>In your case problematic host is not id=5, but id=1. How that host was
>removed?
>
>cloudStack should have been resulted with error when you attempt host
>id=1 deletion while vms are running on it. Right now the system is in
>bad state, and you can't send Stop command for the router as it will
>fail because router is shown as running on removed host, and there is
>no place to send the command to. The only one workaround for situation
>like that - DB modification.
>
>-Alena.
>
>
>
>On 7/16/12 5:38 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>
>>Hi Alena:
>>
>>For example, I removed the host id=5 through the API.
>>
>>Here is the url:
>>http://10.217.5.192:8080/client/api?apikey=iFl88lw1Pk6gKqUIFPN8vzZbJNs
>>U
>>VdY
>>GIJKBTEXtrymcIH5UWp9VHjgnpP_zCmaucmi8XmwK75TR70z-2ayjGA&command=delete
>>H
>>ost
>>&id=dff5e16e-7d8b-47d1-a9a3-8c8e18ca8d23&response=json&signature=CM007
>>f
>>HHr
>>Lsi70cXdMAUIXa6Jbw%3D
>>
>>The JSON output I received was this:
>>
>>{ "deletehostresponse" : { "success" : "true"}  }
>>
>>So, I thought I was good to go.
>>
>>I didn't know to stop a virtual router first. I can do that.
>>Like so?
>>  1) listRouters setting listall=true
>>  2) get the id of the router associated with the host to be deleted.
>>  3) stopRouter id of router from step 2.
>>I need to understand all of the API commands to properly delete a host.
>>
>>Regards,
>>Evan
>>
>>
>>
>>-----Original Message-----
>>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>>Sent: Monday, July 16, 2012 5:26 PM
>>To: cloudstack-users@incubator.apache.org
>>Subject: Re: Problem: Unable to create a deployment for
>>VM[User|i-2-8-VM]
>>
>>How did you remove the host id=1, through the API or UI? It should
>>have been resulted with error because it has vms running on it - your
>>Virtual router r-4-VM.
>>
>>Looks like the system is in a pretty bad state. I can think of one way
>>to recover the vms:
>>
>>1) stop the Management server, save the cloud DB dump.
>>2) In the DB find all Running vms referring the Removed hosts:
>>
>>Select * from vm_instance where state in (Running,Starting, Stopping)
>>and host_id in (select id from host where removed is not null).
>>
>>3) update all these vms with the following info:
>>
>>Update vm_instance set state='Stopped', host_id=null where id in (<ids
>>found on prev step>)
>>
>>4) start the management server
>>5) start the vms updated on step 3).
>>
>>
>>-Alena.
>>
>>
>>
>>On 7/16/12 5:16 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>
>>>Hi Alena:
>>>
>>>It looks like the host table in the database needs to be cleaned up:
>>>
>>>mysql> select id, name, uuid, status from host;
>>>+----+-----------------------------------------------------+---------
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+---
>>>------------------------+--------------+
>>>| id | name                                                | uuid
>>>                        | status       |
>>>+----+-----------------------------------------------------+---------
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+---
>>>------------------------+--------------+
>>>|  1 | xenserver1-cs                                       |
>>>5da2e723-c41a-45bb-9f87-6048c5487ea2 | Removed      |
>>>|  2 | nfs://cumulus.eng.citrite.net/home/export/secondary |
>>>7bf58f58-0cf5-4487-9346-9481fa28ad7a | Alert        |
>>>|  3 | s-1-VM                                              |
>>>780d68d6-f6d7-45de-bbd6-766d8e17ee48 | Disconnected |
>>>|  4 | v-2-VM                                              |
>>>ca98385f-90f5-49ba-8d88-50af5587e7a9 | Disconnected |
>>>|  5 | xenserver1-cs                                       |
>>>dff5e16e-7d8b-47d1-a9a3-8c8e18ca8d23 | Removed      |
>>>|  6 | xenserver1-cs                                       |
>>>7b966f69-e792-4fb5-a5d7-2c3acb294f79 | Up           |
>>>+----+-----------------------------------------------------+---------
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+-
>>>+----+-----------------------------------------------------+---
>>>------------------------+--------------+
>>>6 rows in set (0.00 sec)
>>>
>>>mysql>
>>>
>>>id 1 and id 5 should be deleted.
>>>id 6 has the correct state and uuid.
>>>id 2, 3 and 4 have the wrong state.
>>>
>>>I can fix directly with SQL.
>>>
>>>Any other tables that need fixing besides host?
>>>
>>>Regards,
>>>Evan
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>>>Sent: Monday, July 16, 2012 4:55 PM
>>>To: cloudstack-users@incubator.apache.org
>>>Subject: Re: Problem: Unable to create a deployment for
>>>VM[User|i-2-8-VM]
>>>
>>>The log states that the host doesn't exist:
>>>
>>>com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
>>>unreachable: Host 1: Can't find the host
>>>
>>>
>>>
>>>Can you check the database for host id=1? Make sure its not removed.
>>>
>>>
>>>On 7/16/12 4:45 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>>
>>>>From CSMS GUI, the State is Up for the host, xenserver1-cs.
>>>>
>>>>Regards,
>>>>Evan
>>>>
>>>>
>>>>-----Original Message-----
>>>>From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>>>>Sent: Monday, July 16, 2012 4:45 PM
>>>>To: cloudstack-users@incubator.apache.org
>>>>Subject: Re: Problem: Unable to create a deployment for
>>>>VM[User|i-2-8-VM]
>>>>
>>>>Looks like the Xen host your Virtual Router r-4-VM is running on, is
>>>>in not in UP state. Can you please confirm. If this is the case,
>>>>then the failure is expected. The Xen host should be in UP state.
>>>>
>>>>
>>>>On 7/16/12 4:16 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>>>
>>>>>Running CloudStack Management Server:
>>>>>  v3.0.2.1
>>>>>On:
>>>>>  [root@cumulus management]# uname -a
>>>>>  Linux cumulus.eng.citrite.net 2.6.32-220.el6.x86_64 #1 SMP Tue
>>>>>Dec
>>>>>6
>>>>>19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux
>>>>>  [root@cumulus management]#
>>>>>Hypervisor :
>>>>>  XenServer v6.02
>>>>>
>>>>>Hi:
>>>>>
>>>>>Can¹t add VM instance.
>>>>>Getting this error message from the GUI:
>>>>>  Unable to create a deployment for VM[User|i-2-8-VM]
>>>>>
>>>>>In the attached log file, there are a few ³apparent² issues when
>>>>>trying to create the instance :
>>>>>
>>>>>1)
>>>>>2012-07-16 15:56:19,928 DEBUG
>>>>>[network.router.VirtualNetworkApplianceManagerImpl]
>>>>>(Job-Executor-3:job-33) Applying dhcp entry in network
>>>>>Ntwk[204|Guest|5]
>>>>>2012-07-16 15:56:19,943 WARN
>>>>>[network.router.VirtualNetworkApplianceManagerImpl]
>>>>>(Job-Executor-3:job-33) Unable to apply dhcp entry on disconnected
>>>>>router r-4-VM
>>>>>com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
>>>>>unreachable: Host 1: Can't find the host
>>>>>
>>>>>This one occurred at the beginning of the async job. Not sure if it
>>>>>is an error or not.
>>>>>
>>>>>2)
>>>>>2012-07-16 15:56:20,521 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>>>(Job-Executor-3:job-33) VM state transitted from :Stopped to Error
>>>>>with
>>>>>event: OperationFailedToErrorvm's original host id: null new host id:
>>>>>null host id before state transition:
>>>>> null
>>>>>2012-07-16 15:56:21,192 INFO  [api.commands.DeployVMCmd]
>>>>>(Job-Executor-3:job-33)
>>>>>com.cloud.exception.InsufficientServerCapacityException: Unable to
>>>>>create a deployment for VM[User|i-2-8-VM]Scope=interface
>>>>>com.cloud.dc.DataCenter; id=1
>>>>>2012-07-16 15:56:21,192 WARN  [cloud.api.ApiDispatcher]
>>>>>(Job-Executor-3:job-33) class com.cloud.api.ServerApiException :
>>>>>Unable to create a deployment for VM[User|i-2-8-VM]
>>>>>
>>>>>That one doesn¹t look good.
>>>>>
>>>>>3)
>>>>>2012-07-16 15:56:30,263 DEBUG [cloud.server.StatsCollector]
>>>>>(StatsCollector-1:null) There is no secondary storage VM for
>>>>>secondary storage host
>>>>>nfs://cumulus.eng.citrite.net/home/export/secondary
>>>>>
>>>>>
>>>>>From CloudStack GUI, secondary storage is defined and enabled.
>>>>>
>>>>>What¹s important, which needs to be fixed and how to fix?
>>>>>
>>>>>Thanks.
>>>>>
>>>>>Regards,
>>>>>
>>>>>Evan Miller
>>>>>
>>>>>Citrix Systems. Inc.
>>>>>Desktop and Cloud Engineering Infrastructure
>>>>>4988 Great America Parkway
>>>>>Santa Clara, CA 95054
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>



Mime
View raw message