cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: Unable to take Snapshot CS 4.2.1
Date Wed, 26 Nov 2014 15:26:40 GMT
disclamer - I did not read your logs, but.,,:

CentOS 6.5 version of qemu-img is missing some flag, I beleive  "-s" flag
(redhat removed itd) - and cloudstack still uses that flag...
So if this is also problem in your case - the snapshtos should be created,
but not backup-ed.

Meaning, downgrate qemu-img to CentOs 6.4 version...or use the one in the
folowing way, as described by Lucin / Nux:
http://www.nux.ro/archive/2014/01/Taking_KVM_volume_snapshots_with_Cloudstack_4_2_on_CentOS_6_5.html



On 26 November 2014 at 16:19, Jeff Barnett <jbarnett@prokarma.com> wrote:

> The OS is CentOS 6.5, Storage is NFS not hosted on the controller.  I will
> also note that there is two zones, with the same configuration just using
> different NFS devices one zone can take snapshots but the one that can no
> longer take them has been working for 8 months with no issues.
>
> Here is the section from the log file when I try to take a snapshot:
>
>
> 2014-11-26 07:00:48,308 DEBUG [cloud.api.ApiServlet]
> (catalina-exec-4:null) ===START===  172.16.28.20 -- GET
> command=createSnapshot&volumeid=ec6460c4-896c-46d1-8ae5-75d6431a6608&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014045703
> 2014-11-26 07:00:48,362 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-4:null) submit async job-147 = [
> 2007bb4e-03c9-4ff7-874d-e50641318ace ], details: AsyncJobVO {id:147,
> userId: 2, accountId: 2, sessionKey: null, instanceType: Snapshot,
> instanceId: 128, cmd:
> org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd,
> cmdOriginator: null, cmdInfo:
> {"id":"128","response":"json","sessionkey":"ysbTjq7whHzoePtvcRBRSzsJ6zw\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"2","httpmethod":"GET","_":"1417014045703","volumeid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","ctxAccountId":"2","ctxStartEventId":"5425"},
> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
> processStatus: 0, resultCode: 0, result: null, initMsid: 345048644703,
> completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2014-11-26 07:00:48,363 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ])
> Executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd
> for job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]
> 2014-11-26 07:00:48,363 DEBUG [cloud.api.ApiServlet]
> (catalina-exec-4:null) ===END===  172.16.28.20 -- GET
> command=createSnapshot&volumeid=ec6460c4-896c-46d1-8ae5-75d6431a6608&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014045703
> 2014-11-26 07:00:48,386 INFO  [user.snapshot.CreateSnapshotCmd]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) VOLSS:
> createSnapshotCmd starts:1417014048386
> 2014-11-26 07:00:48,437 DEBUG [agent.transport.Request]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq
> 7-1985284483: Sending  { Cmd , MgmtId: 345048644703, via: 7, Ver: v1,
> Flags: 100011,
> [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02]
> i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"wait":0}}]
> }
> 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq
> 7-1985284483: Executing:  { Cmd , MgmtId: 345048644703, via: 7, Ver: v1,
> Flags: 100011,
> [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02]
> i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"wait":0}}]
> }
> 2014-11-26 07:00:48,438 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-311:null) Seq 7-1985284483: Executing request
> 2014-11-26 07:00:48,438 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-311:null) Seq 7-1985284483: Response Received:
> 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request]
> (DirectAgent-311:null) Seq 7-1985284483: Processing:  { Ans: , MgmtId:
> 345048644703, via: 7, Ver: v1, Flags: 10,
> [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"b7c08601-0654-4749-b7e0-ccc380458e3d","id":0}},"result":true,"wait":0}}]
> }
> 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq
> 7-1985284483: Received:  { Ans: , MgmtId: 345048644703, via: 7, Ver: v1,
> Flags: 10, { CreateObjectAnswer } }
> 2014-11-26 07:00:48,488 DEBUG [storage.motion.AncientDataMotionStrategy]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ])
> copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type
> SNAPSHOT
> 2014-11-26 07:00:48,535 DEBUG [agent.transport.Request]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq
> 9-1742209940: Sending  { Cmd , MgmtId: 345048644703, via: 9, Ver: v1,
> Flags: 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"b7c08601-0654-4749-b7e0-ccc380458e3d","volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02]
> i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/334","volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS
> datastore:
> /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02]
> i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://
> 172.16.25.84/data/CloudStackSEC2","_role":"Image"}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"executeInSequence":true,"wait":21600}}]
> }
> 2014-11-26 07:00:51,404 DEBUG [cloud.api.ApiServlet]
> (catalina-exec-25:null) ===START===  172.16.28.20 -- GET
> command=queryAsyncJobResult&jobId=2007bb4e-03c9-4ff7-874d-e50641318ace&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014048864
> 2014-11-26 07:00:51,428 DEBUG [cloud.api.ApiServlet]
> (catalina-exec-25:null) ===END===  172.16.28.20 -- GET
> command=queryAsyncJobResult&jobId=2007bb4e-03c9-4ff7-874d-e50641318ace&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014048864
> 2014-11-26 07:00:51,840 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) SeqA 3-1302876: Processing Seq 3-1302876:  {
> Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2014-11-26 07:00:51,846 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) SeqA 3-1302876: Sending Seq 3-1302876:  {
> Ans: , MgmtId: 345048644703, via: 3, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2014-11-26 07:00:52,150 DEBUG [agent.transport.Request]
> (AgentManager-Handler-15:null) Seq 9-1742209940: Processing:  { Ans: ,
> MgmtId: 345048644703, via: 9, Ver: v1, Flags: 110,
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"backup
> snapshot exception: Exception: java.lang.Exception\nMessage: unable to
> prepare snapshot backup directory\n","wait":0}}] }
> 2014-11-26 07:00:52,150 DEBUG [agent.transport.Request]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq
> 9-1742209940: Received:  { Ans: , MgmtId: 345048644703, via: 9, Ver: v1,
> Flags: 110, { CopyCmdAnswer } }
> 2014-11-26 07:00:52,155 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-15:null) Seq 9-1742209940: No more commands found
> 2014-11-26 07:00:52,171 DEBUG [storage.snapshot.SnapshotManagerImpl]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Failed
> to create snapshot
> com.cloud.utils.exception.CloudRuntimeException: backup snapshot
> exception: Exception: java.lang.Exception
> Message: unable to prepare snapshot backup directory
>
>         at
> org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280)
>         at
> org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136)
>         at
> org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279)
>         at
> com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1007)
>         at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311)
>         at
> com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2773)
>         at
> org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> 2014-11-26 07:00:52,184 DEBUG [storage.volume.VolumeServiceImpl]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Take
> snapshot: 334 failed
> com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
>         at
> com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1034)
>         at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311)
>         at
> com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2773)
>         at
> org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: com.cloud.utils.exception.CloudRuntimeException: backup
> snapshot exception: Exception: java.lang.Exception
> Message: unable to prepare snapshot backup directory
>
>         at
> org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280)
>         at
> org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136)
>         at
> org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279)
>         at
> com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1007)
>         ... 15 more
> 2014-11-26 07:00:52,184 DEBUG [user.snapshot.CreateSnapshotCmd]
> (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Failed
> to create snapshot
> org.apache.cloudstack.api.ServerApiException
>         at
> org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:176)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> -----Original Message-----
> From: Andrija Panic [mailto:andrija.panic@gmail.com]
> Sent: Tuesday, November 25, 2014 11:55 PM
> To: users@cloudstack.apache.org
> Subject: Re: Unable to take Snapshot CS 4.2.1
>
> Please attach the management logs for further invetigation...what OS, what
> storage, etc...
>
> On 26 November 2014 at 01:51, Jeff Barnett <jbarnett@prokarma.com> wrote:
>
> > When I try to take a snapshot now I just get an error like this,
> > Failed to create a snapshot due to an internal error creating snapshot
> for volume ...
> > has anyone run into this, if so what was the fix, I tried restarting
> > the cloud services on the SSVM, but that did not help.
> >
> >
> >
> >
>
>
> --
>
> Andrija Panić
>



-- 

Andrija Panić

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message