incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chip Childers <chip.child...@sungard.com>
Subject Re: Critical - DO NOT use 4.0 storage clean-up after upgrade from 3.0.2
Date Wed, 14 Nov 2012 16:31:14 GMT
Hi Tamas,

Can you open a bug for this?

On Wed, Nov 14, 2012 at 10:31 AM, Tamas Monos <tamasm@veber.co.uk> wrote:
> Hi,
>
> I had a successful update from 3.0.2->4.0.
> Everything was perfect until I've enabled storage-clean-up in the global options.
>
> At the next first time the storage clean-up thread run it marked all systemVM volumes/templates
on the primary storages for removal.
> It has literally went through all my system VMs and first deleted their volumes than
their templates...
> Then the system start to complain about file-not found messages, deploy failed...
>
> I'm trying to recover from this but I cannot re-create the secondary storage VM due to:
> 2012-11-14 14:17:56,580 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) There
is no secondary storage VM for secondary storage host nfs://192.168.1.2/share/watford_cloud
>
> But I end up with:
> 2012-11-14 14:19:55,241 DEBUG [utils.script.Script] (catalina-exec-6:null) Looking for
createvolume.sh in /usr/lib64/cloud/common/scripts/storage/secondary/createvolume.sh
> 2012-11-14 14:19:55,241 INFO  [storage.template.DownloadManagerImpl] (catalina-exec-6:null)
createvolume.sh found in /usr/lib64/cloud/common/scripts/storage/secondary/createvolume.sh
> 2012-11-14 14:19:55,243 DEBUG [storage.resource.NfsSecondaryStorageResource] (catalina-exec-6:null)
Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy
> 2012-11-14 14:19:55,304 DEBUG [storage.resource.NfsSecondaryStorageResource] (catalina-exec-6:null)
Exit value is 1
> 2012-11-14 14:19:55,304 DEBUG [storage.resource.NfsSecondaryStorageResource] (catalina-exec-6:null)
ln: creating symbolic link `/var/www/html/copy': No such file or directory
> 2012-11-14 14:19:55,304 WARN  [storage.resource.NfsSecondaryStorageResource] (catalina-exec-6:null)
Error in linking  err=ln: creating symbolic link `/var/www/html/copy': No such file or directory
> 2012-11-14 14:19:55,305 INFO  [cloud.resource.ResourceManagerImpl] (catalina-exec-6:null)
Unable to fully initialize the agent because no StartupCommands are returned
> 2012-11-14 14:19:55,305 INFO  [cloud.resource.ResourceManagerImpl] (catalina-exec-6:null)
server resources successfully discovered by SecondaryStorage
> 2012-11-14 14:19:55,305 WARN  [cloud.api.ApiDispatcher] (catalina-exec-6:null) class
com.cloud.api.ServerApiException : Failed to add secondary storage
>
> This is issue is likely caused by the update procedure due to the systemVM were not using
the latest template.
> They were not re-deployed just re-started with the new systemVM template and possible
some references were not updated in other DB tables so the clean-up script found them as not
used and destroyed them all.
> I'm lucky enough I always have test environment for everything.
>
> Regards
>
> Tamas Monos                                               DDI         +44(0)2034687012
> Chief Technical                                             Office    +44(0)2034687000
> Veber: The Hosting Specialists               Fax         +44(0)871 522 7057
> http://www.veber.co.uk<http://www.veber.co.uk/>
>
> Follow us on Twitter: www.twitter.com/veberhost<http://www.twitter.com/veberhost>
> Follow us on Facebook: www.facebook.com/veberhost<http://www.facebook.com/veberhost>
>

Mime
View raw message