cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prachi Damle <Prachi.Da...@citrix.com>
Subject RE: startVirtualMachine fails on StorageUnavailableException
Date Wed, 18 Jul 2012 18:22:17 GMT
Hi Donal,

The deployment planner reuses the pool of a READY ROOT volume and tries to deploy VM to the
same cluster.  Since the ROOT Volume is not in READY state, planner assigns new pools to the
VM's volumes.
Thus state of the ROOT volume drives the pool assignment.

Here since the DATA volume  is already in READY state and created on some other pool than
the pool assigned by planner. Deployment fails while trying to create it on new pool since
it is not recreatable.

Following might be a workaround for this situation:
1. Detach the DATA volume from the VM (VM needs to be in stopped state)
2. Start the VM (This will create the ROOT volume and put it in READY state)
3. Attach the DATA volume (to the running VM or after it is stopped) 

I have logged following issue http://bugs.cloudstack.org/browse/CS-15621 to see if we can
improve the planner strategy by also considering data volume state even if ROOT volume is
not READY.
 
Thanks,
Prachi

-----Original Message-----
From: Donal Lafferty [mailto:donal.lafferty@citrix.com] 
Sent: Wednesday, July 18, 2012 1:37 AM
To: cloudstack-users@incubator.apache.org
Subject: RE: startVirtualMachine fails on StorageUnavailableException

Hi Prachi ,

The DATADISK appears ready according to the queries shown below.  

At the time I'd encountered this problem, I had created 10 volumes, detached them, and added
them to 10 new VMs.  The problem was encountered on all 10 VMs.  Could the DATADISKs have
fallen out of ready while it was transferred to the VMs deployment cluster?

DL

 
[root@cs1-mgr management]# mysql -u cloud -p
mysql> use cloud;

Database changed
mysql> SELECT pool_id, state, volume_type, instance_id FROM `cloud`.`volumes` where id
= 629 ;
+---------+-------+-------------+-------------+
| pool_id | state | volume_type | instance_id |
+---------+-------+-------------+-------------+
|     200 | Ready | DATADISK    |         192 |
+---------+-------+-------------+-------------+
1 row in set (0.00 sec)

mysql> SELECT name,status FROM `cloud`.`storage_pool` where id = 200 ;
+----------------+--------+
| name           | status |
+----------------+--------+
| PrimaryStorage | Up     |
+----------------+--------+
1 row in set (0.00 sec)

mysql> SELECT pool_id, state, volume_type, instance_id FROM `cloud`.`volumes` where id
= 633 ;
+---------+-----------+-------------+-------------+
| pool_id | state     | volume_type | instance_id |
+---------+-----------+-------------+-------------+
|    NULL | Allocated | ROOT        |         192 |
+---------+-----------+-------------+-------------+
1 row in set (0.00 sec)

mysql> SELECT name,status FROM `cloud`.`storage_pool` where id = NULL ;
Empty set (0.00 sec)


-----Original Message-----
From: Prachi Damle [mailto:Prachi.Damle@citrix.com] 
Sent: 17 July 2012 22:43
To: cloudstack-users@incubator.apache.org
Subject: RE: startVirtualMachine fails on StorageUnavailableException

>From the logs, it looks like the data volume of this VM already has a pool assigned. But
since deployment planner did not pick it up for reuse, they may not be in READY state Need
to know what state the volumes for this VM are in and also the status of the pool to which
they are assigned to.


Can you provide the results for following queries over your database:

Data volume:
1) SELECT pool_id, state, volume_type, instance_id FROM `cloud`.`volumes` where id = 629 ;

2) SELECT name,status FROM `cloud`.`storage_pool` where id = <pool id selected above>;

Root volume:
3) SELECT pool_id, state, volume_type, instance_id FROM `cloud`.`volumes` where id = 633 ;

4) SELECT name,status FROM `cloud`.`storage_pool` where id = <pool id selected above>;

Thanks,
Prachi

-----Original Message-----
From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
Sent: Tuesday, July 17, 2012 12:56 PM
To: cloudstack-users@incubator.apache.org
Subject: RE: startVirtualMachine fails on StorageUnavailableException

Hi Prachi,

The snippet for job-2384 can be found downloaded from https://citrix.sharefile.com/d/s0e5a844f0e84dd1a


It's a bit long for cut and paste, but here's a download link that will remain active for
a year.

DL


-----Original Message-----
From: Prachi Damle [mailto:Prachi.Damle@citrix.com]
Sent: 17 July 2012 18:53
To: cloudstack-users@incubator.apache.org
Subject: RE: startVirtualMachine fails on StorageUnavailableException

Hi Donal,

Can you provide entire snippet for the job-2384? Looks like a new pool got assigned to the
already created data disk and since it is not recreatable, the new pool cannot be used.
The log snippet might help showing why a new pool is assigned.

-Prachi

-----Original Message-----
From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
Sent: Tuesday, July 17, 2012 9:35 AM
To: cloudstack-users@incubator.apache.org
Subject: startVirtualMachine fails on StorageUnavailableException

Using a long running CloudStack 3.0.3 deployment, I came up against a StorageUnavailableException
error when starting a VM that had been created in the stopped state.

Any suggestions on how to narrow down the problem further?


Snippet from management server logs:


2012-07-17 12:04:19,107 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-81:job-2384)
Checking if we need to prepare 2 volumes for VM[User|1920fcd5-1633-4af8-9153-d48a5da94e1b]
2012-07-17 12:04:19,107 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-81:job-2384)
Volume Vol[629|vm=192|DATADISK] is not recreatable! Cannot recreate on storagepool: Pool[202|NetworkFilesystem]
2012-07-17 12:04:19,108 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-81:job-2384)
Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [Volume:629] is unreachable: Volume
is not recreatable, Unable to create Vol[629|vm=192|DATADISK]
        at com.cloud.storage.StorageManagerImpl.prepare(StorageManagerImpl.java:3202)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:745)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:462)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2809)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:1785)
        at com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:32)
        at com.cloud.api.commands.StartVMCmd.execute(StartVMCmd.java:119)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:134)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:427)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

Mime
View raw message