cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-3498) [Object_Store_Refactor] Deleting and adding back the zone does not create NFS Cache store and Failure in Sytem VMs Start Up
Date Thu, 18 Jul 2013 23:30:48 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713083#comment-13713083
] 

ASF subversion and git services commented on CLOUDSTACK-3498:
-------------------------------------------------------------

Commit c236645d9a9c03de28a107a5f41bd8399354a802 in branch refs/heads/4.2 from [~minchen07]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=c236645 ]

CLOUDSTACK-3498:[Object_Store_Refactor] Deleting and adding back the
zone does not create NFS Cache store and Failure in Sytem VMs Start Up

                
> [Object_Store_Refactor] Deleting and adding back the zone does not create NFS Cache store
and Failure in Sytem VMs Start Up
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-3498
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3498
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>         Environment: Latest build from ACS 4.2 branch.
>            Reporter: Sanjeev N
>            Assignee: Min Chen
>            Priority: Critical
>             Fix For: 4.2.0
>
>         Attachments: management-server.rar
>
>
> Deleting a zone does not delete the NFS Cache store in that zone and adding another zone
with same NFS cache store details those were given in the deleted zone does not create another
NFS Cache store in the new zone. Hence the Failure in System vms bring up.
> Steps to Reproduce:
> ================
> 1.Bring up CS with advanced zone say zone1.
> 2.Add secondary storage with S3 provider and with NFS Cache Store
> 3.Delete secondary storage
> 4.Delete zone1
> 5.Add another zone say zone2
> 6.Add secondary storage with s3 provider and NFS Cache storage in Zone2 with NFS Server
and Path same as zone1
> Observations:
> ===========
> 1.Deleting zone did not delete NFS Cache store from image_store table.
> 2.Adding NFS Cache store with same details in new zone did not create another entry in
image_store table for the new zone.
> Impact:
> ======
> CS downloads Routing template from S3 to NFS Cache store to bring up the System VMs.
Since CS did not find NFS Cache store in the new zone to copy the routing template spinning
system vms failed with following exception:
> 2013-07-12 06:28:14,731 DEBUG [storage.motion.AncientDataMotionStrategy] (consoleproxy-1:null)
needCacheStorage true, dest at null dest role Primarytemplate/tmpl/1/3/routing-3/systemvmtemplate-2013-06-25-master-kvm.qcow2.bz2
src role Image
> 2013-07-12 06:28:14,737 DEBUG [network.guru.PublicNetworkGuru] (secstorage-1:null) public
network deallocate network: networkId: 210, ip: 10.147.48.3
> 2013-07-12 06:28:14,739 DEBUG [cache.allocator.StorageCacheRandomAllocator] (consoleproxy-1:null)
Can't find cache storage in zone: 3
> 2013-07-12 06:28:14,739 DEBUG [storage.motion.AncientDataMotionStrategy] (consoleproxy-1:null)
copy object failed: com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in
scope id 3 type ZONE
> 2013-07-12 06:28:14,739 DEBUG [storage.motion.AncientDataMotionStrategy] (consoleproxy-1:null)
copy failed
> com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
No cache DataStore in scope id 3 type ZONE
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:204)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:351)
>         at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
>         at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
>         at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-12 06:28:14,752 DEBUG [cloud.storage.VolumeManagerImpl] (consoleproxy-1:null)
Unable to create Vol[91|vm=79|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
No cache DataStore in scope id 3 type ZONE
> 2013-07-12 06:28:14,752 INFO  [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:null)
Unable to contact resource.
> com.cloud.exception.StorageUnavailableException: Resource [StoragePool:3] is unreachable:
Unable to create Vol[91|vm=79|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException:
No cache DataStore in scope id 3 type ZONE
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2493)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
>         at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
>         at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-12 06:28:14,758 DEBUG [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:null)
Cleaning up resources for the vm VM[ConsoleProxy|v-79-VM] in Starting state
> DB entries from Cloud table:
> ======================
> mysql> select * from image_store where role='Image' and removed is null\G;
> *************************** 1. row ***************************
>                  id: 3
>                name: objstore
> image_provider_name: S3
>            protocol: http
>                 url: NULL
>      data_center_id: NULL
>               scope: REGION
>                role: Image
>                uuid: 0c444304-7e5f-4137-8da1-a812c3b81cf7
>              parent: NULL
>             created: 2013-07-11 15:07:56
>             removed: NULL
>          total_size: NULL
>          used_bytes: NULL
> *************************** 2. row ***************************
>                  id: 5
>                name: obj_store
> image_provider_name: S3
>            protocol: http
>                 url: NULL
>      data_center_id: NULL
>               scope: REGION
>                role: Image
>                uuid: 789b6388-ea95-41e6-98e6-e2ded85f119f
>              parent: NULL
>             created: 2013-07-12 10:23:09
>             removed: NULL
>          total_size: NULL
>          used_bytes: NULL
> 2 rows in set (0.00 sec)
> mysql> select * from image_store where role='ImageCache' and removed is null\G;
> *************************** 1. row ***************************
>                  id: 1
>                name: nfs://10.147.28.7/export/home/sanjeev/sec_xen_os
> image_provider_name: NFS
>            protocol: nfs
>                 url: nfs://10.147.28.7/export/home/sanjeev/sec_xen_os
>      data_center_id: 1
>               scope: ZONE
>                role: ImageCache
>                uuid: 8e43fa4b-cf0c-46c2-9e68-ead6bd481742
>              parent: NULL
>             created: 2013-07-11 14:51:18
>             removed: NULL
>          total_size: NULL
>          used_bytes: NULL
> *************************** 2. row ***************************
>                  id: 4
>                name: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
> image_provider_name: NFS
>            protocol: nfs
>                 url: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
>      data_center_id: 2
>               scope: ZONE
>                role: ImageCache
>                uuid: b42417d4-3e97-4f75-9439-25f97badcaa1
>              parent: NULL
>             created: 2013-07-11 16:41:51
>             removed: NULL
>          total_size: NULL
>          used_bytes: NULL
> 2 rows in set (0.00 sec)
> mysql> select * from data_center;
> +----+---------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+-----------------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------------------+-------+---------+-------------+--------------+----------+----------+
> | id | name    | uuid                                 | description | dns1          |
dns2 | internal_dns1 | internal_dns2 | gateway | netmask | router_mac_address | mac_address
| guest_network_cidr | domain | domain_id | networktype | dns_provider  | gateway_provider
| firewall_provider | dhcp_provider | lb_provider           | vpn_provider  | userdata_provider
| allocation_state | zone_token                           | is_security_group_enabled | is_local_storage_enabled
| removed             | owner | created | lastUpdated | engine_state | ip6_dns1 | ip6_dns2
|
> +----+---------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+-----------------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------------------+-------+---------+-------------+--------------+----------+----------+
> |  1 | Basic   | d234e7a5-5cd0-4029-9b80-07a47f470e79 | NULL        | 10.103.128.16 |
NULL | 10.103.128.16 | NULL          | NULL    | NULL    | 02:00:00:00:00:01  |          21
| NULL               | NULL   |      NULL | Basic       | VirtualRouter | NULL           
 | NULL              | VirtualRouter | ElasticLoadBalancerVm | NULL          | VirtualRouter
    | Enabled          | 64c45795-5345-36e5-b52d-5530cd602a55 |                         1
|                        0 | NULL                | NULL  | NULL    | NULL        | Disabled
    | NULL     | NULL     |
> |  2 | NULL    | 45072eac-10d8-499d-9325-d4ee79e45026 | NULL        | 10.103.128.16 |
NULL | 10.103.128.16 | NULL          | NULL    | NULL    | 02:00:00:00:00:01  |          19
| 10.1.1.0/24        | NULL   |      NULL | Advanced    | VirtualRouter | VirtualRouter  
 | VirtualRouter     | VirtualRouter | VirtualRouter         | VirtualRouter | VirtualRouter
    | Disabled         | b2359ba6-39c7-3dc6-a2ca-ac9091a6a06b |                         0
|                        1 | 2013-07-12 10:09:14 | NULL  | NULL    | NULL        | Disabled
    | NULL     | NULL     |
> |  3 | Adv-kvm | 37b35302-af3c-43b9-bc6c-acb17642b0d3 | NULL        | 10.103.128.16 |
NULL | 10.103.128.16 | NULL          | NULL    | NULL    | 02:00:00:00:00:01  |          21
| 10.1.1.0/24        | NULL   |      NULL | Advanced    | VirtualRouter | VirtualRouter  
 | VirtualRouter     | VirtualRouter | VirtualRouter         | VirtualRouter | VirtualRouter
    | Disabled         | 8f2a7a84-dbfc-35e0-9201-f3066c65b414 |                         0
|                        1 | NULL                | NULL  | NULL    | NULL        | Disabled
    | NULL     | NULL     |
> +----+---------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+-----------------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------------------+-------+---------+-------------+--------------+----------+----------+
> 3 rows in set (0.00 sec)
> 1.Data_Center 2 was removed from the setup, however we can see the image store with role
image_cache for data_center 2 in the image_store table.
> 2. Data_Center 3 was created after deleting DC 2 , and provided the same image_store
details as DC2, hence there is no image store with role image_cache for DC3.
>  
>   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message