We found this issue in our testing as well. The reason is template_spool_ref table is not getting
updated with correct template_size
So we see this error "Unable to start instance due to can't find mapping in ObjectInDataStore
table for: org.apache.cloudstack.storage.image.store.TemplateObject@16352d04"
This is a product issue and needs to be fixed.
-Sanjeev
-----Original Message-----
From: Daniel Kollmer [mailto:Daniel.Kollmer@tomtom.com]
Sent: Wednesday, March 25, 2015 9:38 PM
To: users@cloudstack.apache.org
Subject: Can not deploy system VMs anymore
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello All
This is related to my mail from yesterday:
> Today I wanted to try and build a new zone (on cloudstack 4.5/KVM)
> with VXLAN but upon launching it appears to be unable to start the
> systemVMs
>
> I constantly got errors of this kind:
>
> Exception while trying to start secondary storage vm
> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
> unreachable: Host 1: Unable to start instance due to Unable to acquire
> lock on VMTemplateStoragePool:
>
> I have verified that I can mount the secondary storage from the
> hypervisor, it works.
>
> I tried reverting the zone back to a setup which does not use VXLAN
> but the result was the same. If I check on the hypervisor itself I see
> messages like this repeating over and over again:
>
>
> 2015-03-24 17:46:28,373{GMT} INFO
> [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:)
> Attempting to remove storage pool 5a7d9fa7-3f8a-3e91-802e-bee6c7a2654c
> from libvirt ^@2015-03-24 17:46:55,195{GMT} INFO
> [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-3:) Attempting to create storage pool
> 5a7d9fa7-3f8a-3e91-802e-bee6c7a2654c (NetworkFilesystem) in libvirt
> 2015-03-24 17:46:55,292{GMT} INFO [kvm.storage.LibvirtStorageAdaptor]
> (agentRequest-Handler-3:) Attempting to create volume
> ea985424-d237-11e4-b2aa-c81f66f5bb98
> (NetworkFilesystem) in pool 9af29c7c-b3b6-329b-8427-8929fb18fedf
> with size 2684354560
>
> I have the feeling something is going wrong there with the libvirt
> setup. I have completely emptied the secondary storage and re-deployed
> the secondary storage VM but to no avail.
>
I have tried all day to find out what is going on there. I have destroyed and recreated everything
from scratch including the database and SystemVM seeding. I have reverted to a plain setup
without OVS or VXLAN and just one layer 2 network.
All machines can reach each other, mount the NFS storage etc, but I still get the same errors.
Please see the full relevant log here http://pastebin.com/DGLTYrjQ
I really am at my wits end because that just stopped working from one day to the next and
I have no idea why.
I hope someone can point me into the right direction.
- --
Daniel Kollmer | Sr. System Engineer - OTS | daniel.kollmer@tomtom.com | Phone: +31 20 75
75 084 | Mobile: +31
(0)6 29 13 96 47
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBAgAGBQJVEt10AAoJEAFpCdELSZ7Z+DUP/R8fEWk4DYtiAWr4hkrIM1Wy
m353Unmjv1Whv9IvdBLodTUHFSHMHtOyc6+hpkr8LyhXy+0S0zWqtXZJ5cwA6ndZ
WmrTvQOk65qTkhpni+t3W1s3dWKbBEfoMNX5L1jDCIzYU516KUlmkxaQYmJVOpK9
RTlI2HFDqOpiCJ9QswcecvYpXUA/2wiVp/FgNqSwgDzPc2vnzxgo+5AGtcfF4Zwm
4eaTF4GEEeciidZv2x/gCCTXoCEHv4x0uDX96O3w0MKRu/BgjkT8X6u4XCZBTFCy
3pTAtswcZYukL0LOESI5+G95MeRfvGG72uZyk6DMU2/Qhh2t4d/raZmq0dsf6itk
Nj5vdfHqc9x0O9e6g1U8mgcaA/NCXLhBpHnzN4EYLO/O3THBz8pPBTZ0Ajz7jHgT
Fu1D9uq8YR6mAaeIyI5MT5QjNmfZGeAPWxEgCHaBJ0kOKGzGTFzREsVd1qFe48BE
RqmpZUV2lXvCDLMU6Ml9GMIKsRoR3snW1WBExd03FScafKNsQBCJ+Y5FDNM4+w6I
mz1Le+DP2IPxjBNPkaUUXC/tTgx1L1ET/PGegCESFnZcjTrTEi+yrwD6oomFvvx/
h6BZpdkiw3eQr6KDV6csZT3CdX2PO5LWRC2xIP6domfDYKzLOw/4SQxVLDwkYIXk
JGqEFEm30TJya7GzBMyI
=YE3k
-----END PGP SIGNATURE-----
|