cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joris van Lieshout (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-692) The CleanupSnapshotBackup process on SSVM deletes snapshots that are still in the process of being created.
Date Tue, 17 Sep 2013 13:55:51 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joris van Lieshout updated CLOUDSTACK-692:
------------------------------------------

    Summary: The CleanupSnapshotBackup process on SSVM deletes snapshots that are still in
the process of being created.  (was: The StorageManager-Scavenger deletes snapshots that are
still in the process of being created.)
    
> The CleanupSnapshotBackup process on SSVM deletes snapshots that are still in the process
of being created.
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-692
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-692
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Snapshot
>            Reporter: Joris van Lieshout
>            Priority: Minor
>
> Hi there,
> I think we ran into a bug due to a concurrence of circumstances regarding snapshotting
and the cleanup of snapshots.
> The CleanupSnapshotBackup process on the SSVM deletes vhd files that are not known in
the database but when, especially long running snapshot, are being copied to secondary storeage
there is a gap between the start and finish of the VDI-copy, where the uuid of the destination
vhd is not registered in the database. If the CleanupSnapshotBackup deletes the destinaion
vhd during this window it results in hanging sparse_dd process on the XenServer hypervisor
pointing to a tapdisk2 process with no file behind it.
> ===Secondary storage vm (2 hour time difference due to time zone). second to last line
you see the vhd being deleted.
> 2013-09-04 03:14:45,721 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Request:Seq
261-1870805144:  { Cmd , MgmtId: 345052370018, via: 261, Ver: v1, Flags: 100011, [{"CleanupSnapshotBackupCommand":{"secondaryStoragePoolURL":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"validBackupUUIDs":["1a56760b-d1c0-4620-8cf7-271951500d70","b6157bc9-085b-4ed6-95c2-4341f31c64bf","1ff967e3-3606-4112-9155-b1145b2ef576","12fbe4e3-1fdd-4c35-a961-0fce07cff584","278e9915-4f94-40c8-bef4-9c6bc82d4653","6fba1dd7-4736-47b3-9eed-148304c0e192","b9d8c9d8-6445-463b-b4e1-ab3b3f3a67a2","40ba5d72-c69a-46c2-973b-0570c1cabeac","774f2b0e-cdaf-4594-a9f9-4f872dcaad6e","8269f50b-6bec-427c-8186-540df6a75dbf","7b0c6e75-40cf-4dd7-826a-09b39f3da7b5","df7eac9c-137a-4655-9d21-d781916351f1","11ec2db1-a2fc-4221-ae1a-c1ab2bd59509","dfc348e1-af50-4d77-b4a0-6e86fc954e1c","98f64c0f-7498-4c70-8b70-beaefd723b45","c42f9dd5-079d-4b77-86dc-c19b7fbed817"],"wait":0}}]
}
> 2013-09-04 03:14:45,722 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Processing
command: com.cloud.agent.api.CleanupSnapshotBackupCommand
> 2013-09-04 03:14:45,723 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:)
Executing: mount 
> 2013-09-04 03:14:45,732 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:)
Execution is successful.
> 2013-09-04 03:14:45,772 WARN  [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:)
snapshot 8ca9fea4-8a98-4cc3-bba7-cc1dcf32bb24.vhd is not recorded in DB, remove it
> 2013-09-04 03:14:45,772 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 261-1870805144:
 { Ans: , MgmtId: 345052370018, via: 261, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}]
}
> ==== management-server.log. here you see the snapshot being created, the copyToSecStorage
process starting, eventually timing out due to the hanging vdi-copy, failing on retrying because
vdi in use (although not existing any more the vdi is still know on xen), retrying some more
on another HV and eventuall giving up because it tries to create a duplicate SR.
> 2013-09-04 04:27:10,931 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-69:job-95137)
Executing com.cloud.api.commands.CreateSnapshotCmd for job-95137
> 2013-09-04 04:27:10,971 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303147: Sending  { Cmd , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 100011, [{"ManageSnapshotCommand":{"_commandSwitch":"-c","_volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","_pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"_snapshotName":"vlstws3_ROOT-2736_20130904022710","_snapshotId":71889,"_vmName":"i-45-2736-VM","wait":0}}]
}
> 2013-09-04 04:27:10,971 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303147: Executing:  { Cmd , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 100011, [{"ManageSnapshotCommand":{"_commandSwitch":"-c","_volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","_pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"_snapshotName":"vlstws3_ROOT-2736_20130904022710","_snapshotId":71889,"_vmName":"i-45-2736-VM","wait":0}}]
}
> 2013-09-04 04:27:12,949 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303147: Received:  { Ans: , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 10, { ManageSnapshotAnswer
} }
> 2013-09-04 04:27:12,991 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303148: Sending  { Cmd , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 04:27:12,991 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303148: Executing:  { Cmd , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 04:27:12,992 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 91-780303148: Executing request
> 2013-09-04 04:27:13,561 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Creating a not shared SR for nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1/snapshots/45/5863
> 2013-09-04 04:27:14,499 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Checking 00e04f60-4c80-37e4-9abe-855b16fbc9e1 or SR a4ca3b8c-6e6d-5f07-cb71-f6c9b7f950cb on
XS[d868bf32-d747-4ec0-b1c2-fba3df1c85bd-192.168.52.15]
> 2013-09-04 04:27:14,512 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Host 192.168.52.15 OpaqueRef:5ee9850b-b15c-b39b-dd27-ff226a1b3126: Created a SR; UUID is a4ca3b8c-6e6d-5f07-cb71-f6c9b7f950cb
device config is {serverpath=/volumes/pool0/MCCP-SHARED-1-1/snapshots/45/5863, server=mccpnas7.storage.mccp.mcinfra.net}
> 2013-09-04 10:27:12,992 DEBUG [agent.manager.AgentAttache] (Job-Executor-69:job-95137)
Seq 91-780303148: Waiting some more time because this is the current command
> 2013-09-04 10:27:34,262 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Async 21600 seconds timeout for task com.xensource.xenapi.Task@ec1aac0e
> 2013-09-04 10:27:34,585 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Host 192.168.52.15 OpaqueRef:5ee9850b-b15c-b39b-dd27-ff226a1b3126: Removing SR
> 2013-09-04 10:27:34,624 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Host 192.168.52.15 OpaqueRef:5ee9850b-b15c-b39b-dd27-ff226a1b3126: Catch XenAPIException:
This operation cannot be performed because this VDI is in use by some other operation
> 2013-09-04 10:27:34,650 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Host 192.168.52.15 OpaqueRef:5ee9850b-b15c-b39b-dd27-ff226a1b3126: Catch XenAPIException:
This operation cannot be performed because this VDI is in use by some other operation
> 2013-09-04 10:27:34,650 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
Host 192.168.52.15 OpaqueRef:5ee9850b-b15c-b39b-dd27-ff226a1b3126: Unable to remove SR
> 2013-09-04 10:27:34,650 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-441:null)
BackupSnapshot Failed due to Async 21600 seconds timeout for task com.xensource.xenapi.Task@ec1aac0e
> 2013-09-04 10:27:34,651 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 91-780303148: Response Received: 
> 2013-09-04 10:27:34,651 DEBUG [agent.transport.Request] (DirectAgent-441:null) Seq 91-780303148:
Processing:  { Ans: , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 10, [{"BackupSnapshotAnswer":{"full":true,"result":false,"details":"BackupSnapshot
Failed due to Async 21600 seconds timeout for task com.xensource.xenapi.Task@ec1aac0e","wait":0}}]
}
> 2013-09-04 10:27:34,651 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
91-780303148: Received:  { Ans: , MgmtId: 345052370017, via: 91, Ver: v1, Flags: 10, { BackupSnapshotAnswer
} }
> 2013-09-04 10:27:34,651 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Async
21600 seconds timeout for task com.xensource.xenapi.Task@ec1aac0e through 91
> 2013-09-04 10:29:34,652 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Retrying...
> 2013-09-04 10:29:34,659 DEBUG [agent.manager.ClusteredAgentAttache] (Job-Executor-69:job-95137)
Seq 89-1251159458: Forwarding Seq 89-1251159458:  { Cmd , MgmtId: 345052370017, via: 89, Ver:
v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
} to 345052370018
> 2013-09-04 10:29:43,085 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
89-1251159458: Received:  { Ans: , MgmtId: 345052370017, via: 89, Ver: v1, Flags: 10, { BackupSnapshotAnswer
} }
> 2013-09-04 10:29:43,086 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Can
not create second storage SR mountpoint: mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1/snapshots/45/5863
due to SR_BACKEND_FAILURE_40The SR scan failed  [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI",
"uuid", "df7eac9c-137a-4655-9d21-d781916351f1")']] through 89
> 2013-09-04 10:30:17,489 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Ping from 109
> 2013-09-04 10:31:18,345 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 95-1001324546: Executing request
> 2013-09-04 10:31:18,642 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 95-1001324546: Response Received: 
> 2013-09-04 10:31:18,642 DEBUG [agent.transport.Request] (DirectAgent-441:null) Seq 95-1001324546:
Processing:  { Ans: , MgmtId: 345052370017, via: 95, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}]
}
> 2013-09-04 10:31:19,423 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 41-1598291970: Executing request
> 2013-09-04 10:31:19,663 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-441:null)
Seq 41-1598291970: Response Received: 
> 2013-09-04 10:31:19,663 DEBUG [agent.transport.Request] (DirectAgent-441:null) Seq 41-1598291970:
Processing:  { Ans: , MgmtId: 345052370017, via: 41, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}]
}
> 2013-09-04 10:31:43,086 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Retrying...
> 2013-09-04 10:31:43,091 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
93-156960768: Sending  { Cmd , MgmtId: 345052370017, via: 93, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 10:31:43,091 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
93-156960768: Executing:  { Cmd , MgmtId: 345052370017, via: 93, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 10:31:56,375 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
93-156960768: Received:  { Ans: , MgmtId: 345052370017, via: 93, Ver: v1, Flags: 10, { BackupSnapshotAnswer
} }
> 2013-09-04 10:31:56,376 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Can
not create second storage SR mountpoint: mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1/snapshots/45/5863
due to SR_BACKEND_FAILURE_40The SR scan failed  [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI",
"uuid", "df7eac9c-137a-4655-9d21-d781916351f1")']] through 93
> 2013-09-04 10:33:56,376 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Retrying...
> 2013-09-04 10:33:56,381 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
95-1001356346: Sending  { Cmd , MgmtId: 345052370017, via: 95, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 10:33:56,381 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
95-1001356346: Executing:  { Cmd , MgmtId: 345052370017, via: 95, Ver: v1, Flags: 100011,
[{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-45-2736-VM","snapshotId":71889,"pool":{"id":208,"uuid":"b290385b-466d-3243-a939-3d242164e034","host":"mccpnas3-4-vip1.mccp.mcinfra.net","path":"/volumes/pool0/MCCP-S-SBP1-1_MCCP-XEN-1","port":2049,"type":"NetworkFilesystem"},"primaryStoragePoolNameLabel":"b290385b-466d-3243-a939-3d242164e034","snapshotUuid":"6396ec81-4128-44ca-a121-d9f1bf789d6c","snapshotName":"vlstws3_ROOT-2736_20130904022710","secondaryStorageUrl":"nfs://mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1","dcId":1,"accountId":45,"volumeId":5863,"volumePath":"9cb7af90-ca88-4b34-aa6f-bc21c3d4a3aa","wait":21600}}]
}
> 2013-09-04 10:34:13,682 DEBUG [agent.transport.Request] (Job-Executor-69:job-95137) Seq
95-1001356346: Received:  { Ans: , MgmtId: 345052370017, via: 95, Ver: v1, Flags: 10, { BackupSnapshotAnswer
} }
> 2013-09-04 10:34:13,683 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Can
not create second storage SR mountpoint: mccpnas7.storage.mccp.mcinfra.net/volumes/pool0/MCCP-SHARED-1-1/snapshots/45/5863
due to SR_BACKEND_FAILURE_40The SR scan failed  [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI",
"uuid", "df7eac9c-137a-4655-9d21-d781916351f1")']] through 95
> 2013-09-04 10:36:13,683 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Retrying...
> 2013-09-04 10:36:13,685 WARN  [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Storage unavailable 
> 2013-09-04 10:36:13,685 WARN  [storage.snapshot.SnapshotManagerImpl] (Job-Executor-69:job-95137)
Failed to back up snapshot on secondary storage, deleting the record from the DB
> 2013-09-04 10:36:13,702 ERROR [cloud.api.ApiDispatcher] (Job-Executor-69:job-95137) Exception
while executing CreateSnapshotCmd:
> 2013-09-04 10:36:13,702 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-69:job-95137)
Complete async job-95137, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text:
Created snapshot: com.cloud.storage.SnapshotVO$$EnhancerByCGLIB$$3173e384@40abfe02 on primary
but failed to backup on secondary

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message