cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjeev N (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-3667) [Object_Store_Refactor][VMWare] System VMs StartCommand failed due to Exception: java.lang.Exception Message: secondary storage for dc 3 is not ready yet?
Date Fri, 19 Jul 2013 11:10:49 GMT
Sanjeev N created CLOUDSTACK-3667:
-------------------------------------

             Summary: [Object_Store_Refactor][VMWare] System VMs StartCommand failed due to
Exception: java.lang.Exception Message: secondary storage for dc 3 is not ready yet?
                 Key: CLOUDSTACK-3667
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3667
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.2.0
         Environment: Latest build from ACS 4.2 Branch.
Zone: Advanced with VMWare cluster
Storage: S3 and ISCSI(Primary)
            Reporter: Sanjeev N
            Priority: Blocker
             Fix For: 4.2.0


[Object_Store_Refactor][VMWare] System VMs StartCommand failed due to Exception: java.lang.Exception
Message: secondary storage for dc 3 is not ready yet?

System VMs failed to start in case of VMWare cluster due to NULL in URL for  S3 image store
in image_store table with role Image.

I looked at the following code in incubator-cloudstack/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
:
 // prepare systemvm patch ISO
            if (vmSpec.getType() != VirtualMachine.Type.User) {
                // attach ISO (for patching of system VM)
                String secStoreUrl = mgr.getSecondaryStorageStoreUrl(Long.parseLong(_dcId));
                if(secStoreUrl == null) {
                    String msg = "secondary storage for dc " + _dcId + " is not ready yet?";
                    throw new Exception(msg);
                }
                mgr.prepareSecondaryStorageStore(secStoreUrl);


Since secStoreUrl  is matching with null , system vms starting failing with following exception
message:
java.lang.Exception: secondary storage for dc 3 is not ready yet?

Steps to Reproduce:
================
1.Bring up CS in advanced zone with vmware cluster
2.Add S3 image store and NFS cache store in the zone
3.Enable the zone

Observations:
===========
 2013-07-19 11:50:10,861 WARN  [vmware.resource.VmwareResource] (DirectAgent-97:10.147.40.13)
StartCommand failed due to Exception: java.lang.Exception
Message: secondary storage for dc 3 is not ready yet?

java.lang.Exception: secondary storage for dc 3 is not ready yet?
        at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2628)
        at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:519)
        at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

Enabled TRACE in log4j-cloud.xml and found the following sql statements executed before the
exception:
mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol,
image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope,
image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size,
image_store.used_bytes FROM image_store WHERE image_store.role = 'Image'  AND  (image_store.scope
= 'REGION'  OR image_store.data_center_id = 3 )  AND image_store.removed IS NULL;
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
| id | name       | uuid                                 | protocol | url  | image_provider_name
| data_center_id | scope  | created             | removed | role  | parent | total_size |
used_bytes |
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
|  8 | obj_store  | ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d | http     | NULL | S3          
       |           NULL | REGION | 2013-07-16 12:07:09 | NULL    | Image | NULL   |      
NULL |       NULL |
| 17 | obj_store2 | 9e55b78b-cead-45e4-a84e-b35896d961a8 | http     | NULL | S3          
       |           NULL | REGION | 2013-07-19 15:12:45 | NULL    | Image | NULL   |      
NULL |       NULL |
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
2 rows in set (0.00 sec)

mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol,
image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope,
image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size,
image_store.used_bytes FROM image_store WHERE image_store.id = 17  AND image_store.removed
IS NULL;
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
| id | name       | uuid                                 | protocol | url  | image_provider_name
| data_center_id | scope  | created             | removed | role  | parent | total_size |
used_bytes |
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
| 17 | obj_store2 | 9e55b78b-cead-45e4-a84e-b35896d961a8 | http     | NULL | S3          
       |           NULL | REGION | 2013-07-19 15:12:45 | NULL    | Image | NULL   |      
NULL |       NULL |
+----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
1 row in set (0.00 sec)
mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol,
image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope,
image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size,
image_store.used_bytes FROM image_store WHERE image_store.id = 8  AND image_store.removed
IS NULL;
+----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
| id | name      | uuid                                 | protocol | url  | image_provider_name
| data_center_id | scope  | created             | removed | role  | parent | total_size |
used_bytes |
+----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
|  8 | obj_store | ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d | http     | NULL | S3           
      |           NULL | REGION | 2013-07-16 12:07:09 | NULL    | Image | NULL   |       NULL
|       NULL |
+----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
1 row in set (0.00 sec)

ulr value is set to null incase of S3 storage provider.

Following is the image_store table content:
=================================
mysql> select * from image_store where removed is null\G;
*************************** 1. row ***************************
                 id: 7
               name: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
image_provider_name: NFS
           protocol: nfs
                url: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
     data_center_id: 1
              scope: ZONE
               role: ImageCache
               uuid: aa07c444-910a-43ff-bfb7-621c0ddc6223
             parent: NULL
            created: 2013-07-16 12:07:09
            removed: NULL
         total_size: NULL
         used_bytes: NULL
*************************** 2. row ***************************
                 id: 8
               name: obj_store
image_provider_name: S3
           protocol: http
                url: NULL
     data_center_id: NULL
              scope: REGION
               role: Image
               uuid: ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d
             parent: NULL
            created: 2013-07-16 12:07:09
            removed: NULL
         total_size: NULL
         used_bytes: NULL
*************************** 3. row ***************************
                 id: 16
               name: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
image_provider_name: NFS
           protocol: nfs
                url: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
     data_center_id: 3
              scope: ZONE
               role: ImageCache
               uuid: c3e72ffb-2040-4110-856a-ffabf4ca5992
             parent: NULL
            created: 2013-07-19 15:12:45
            removed: NULL
         total_size: NULL
         used_bytes: NULL
*************************** 4. row ***************************
                 id: 17
               name: obj_store2
image_provider_name: S3
           protocol: http
                url: NULL
     data_center_id: NULL
              scope: REGION
               role: Image
               uuid: 9e55b78b-cead-45e4-a84e-b35896d961a8
             parent: NULL
            created: 2013-07-19 15:12:45
            removed: NULL
         total_size: NULL
         used_bytes: NULL
4 rows in set (0.00 sec)

ERROR:
No query specified

Attached management server log file.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message