cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "France (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-8044) Failed to create snapshot due to an internal error creating snapshot for volume 372 -> Failure from sparse_dd: Fatal error: exception Invalid_argument("index out of bounds")
Date Mon, 08 Dec 2014 09:28:12 GMT
France created CLOUDSTACK-8044:
----------------------------------

             Summary: Failed to create snapshot due to an internal error creating snapshot for volume 372 -> Failure from sparse_dd: Fatal error: exception Invalid_argument("index out of bounds")  
                 Key: CLOUDSTACK-8044
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8044
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.3.1
         Environment: XS 6.0.2+Hotfixes, ACS 4.3.1


            Reporter: France


After upgrading from CS 4.1.1 to CS 4.3.1, one of the instances can not get a snapshot. There were/are some errors for all instances, as described here:
https://issues.apache.org/jira/browse/CLOUDSTACK-8013
but this is currently the only instance, where snapshots are actually not working. This instance has had its root disk size extended and cloud.volumes.size field updated manually in the DB accordingly after the upgrade.
Suresh Babu, has supposedly tried the same procedure, but failed to get the error as we did.

Here is the management server log:
014-12-08 08:46:35,858 DEBUG [c.c.a.ApiServlet] (http-6443-exec-121:ctx-5b80bdcb) ===START===  XX.XX.XX.XX -- GET  command=createSnapshot&volumeid=e1cf5716-4af1-47cf-8f8d-84979ca55183&quiescevm=false&response=json&sessionkey=CENSOREDo%3D&_=1418024978323
2014-12-08 08:46:35,869 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) Access to Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] granted to Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,882 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) Access to org.apache.cloudstack.storage.volume.VolumeObject@6329ffac granted to Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,951 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-20:ctx-f5bed97c) Add job-2834 into job monitoring
2014-12-08 08:46:35,951 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-20:ctx-f5bed97c) Executing AsyncJobVO {id:2834, userId: 43, accountId: 45, instanceType: Snapshot, instanceId: 1224, cmd: org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, cmdInfo: {"id":"1224","response":"json","sessionkey":"CENSOREDo\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"43","httpmethod":"GET","volumeid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","_":"1418024978323","quiescevm":"false","ctxAccountId":"45","ctxStartEventId":"62631"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 95545481387, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2014-12-08 08:46:35,951 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) submit async job-2834, details: AsyncJobVO {id:2834, userId: 43, accountId: 45, instanceType: Snapshot, instanceId: 1224, cmd: org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, cmdInfo: {"id":"1224","response":"json","sessionkey":"CENSOREDo\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"43","httpmethod":"GET","volumeid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","_":"1418024978323","quiescevm":"false","ctxAccountId":"45","ctxStartEventId":"62631"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 95545481387, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2014-12-08 08:46:35,953 DEBUG [c.c.a.ApiServlet] (http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) ===END===  XX.XX.XX.XX -- GET  command=createSnapshot&volumeid=e1cf5716-4af1-47cf-8f8d-84979ca55183&quiescevm=false&response=json&sessionkey=CENSOREDo%3D&_=1418024978323
2014-12-08 08:46:35,959 DEBUG [c.c.u.AccountManagerImpl] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Access to Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] granted to Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,983 INFO  [o.a.c.a.c.u.s.CreateSnapshotCmd] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) VOLSS: createSnapshotCmd starts:1418024795983
2014-12-08 08:46:36,073 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 4-2104434319: Sending  { Cmd , MgmtId: 95545481387, via: 4(x4.c.some.domain), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"wait":0}}] }
2014-12-08 08:46:36,074 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 4-2104434319: Executing:  { Cmd , MgmtId: 95545481387, via: 4(x4.c.some.domain), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"wait":0}}] }
2014-12-08 08:46:36,074 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-55:ctx-c9141c38) Seq 4-2104434319: Executing request
2014-12-08 08:46:36,105 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-26:ctx-2dbaf7f0) Seq 4-2104434318: Response Received: 
2014-12-08 08:46:36,106 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4) Seq 4-2104434318: Received:  { Ans: , MgmtId: 95545481387, via: 4, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:36,115 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-490:ctx-3bb693d5) Seq 18-40520662: Executing request
2014-12-08 08:46:36,323 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-490:ctx-3bb693d5) Seq 18-40520662: Response Received: 
2014-12-08 08:46:36,324 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4) Seq 18-40520662: Received:  { Ans: , MgmtId: 95545481387, via: 18, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:36,331 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-264:ctx-e3747a2c) Seq 20-58412855: Executing request
2014-12-08 08:46:36,657 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-264:ctx-e3747a2c) Seq 20-58412855: Response Received: 
2014-12-08 08:46:36,657 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4) Seq 20-58412855: Received:  { Ans: , MgmtId: 95545481387, via: 20, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:37,471 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-8:null) SeqA 27-899693: Processing Seq 27-899693:  { Cmd , MgmtId: -1, via: 27, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n  \"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:37,503 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-8:null) SeqA 27-899693: Sending Seq 27-899693:  { Ans: , MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:39,022 DEBUG [c.c.a.ApiServlet] (http-6443-exec-119:ctx-a36727f4) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024981486
2014-12-08 08:46:39,051 DEBUG [c.c.a.ApiServlet] (http-6443-exec-119:ctx-a36727f4 ctx-64cd7dcd) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024981486
2014-12-08 08:46:40,081 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-55:ctx-c9141c38) Seq 4-2104434319: Response Received: 
2014-12-08 08:46:40,082 DEBUG [c.c.a.t.Request] (DirectAgent-55:ctx-c9141c38) Seq 4-2104434319: Processing:  { Ans: , MgmtId: 95545481387, via: 4, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}] }
2014-12-08 08:46:40,082 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 4-2104434319: Received:  { Ans: , MgmtId: 95545481387, via: 4, Ver: v1, Flags: 10, { CreateObjectAnswer } }
2014-12-08 08:46:40,140 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type SNAPSHOT
2014-12-08 08:46:40,184 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 2-902104676: Sending  { Cmd , MgmtId: 95545481387, via: 2(x2.c.some.domain), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/45/372","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary","_role":"Image"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}] }
2014-12-08 08:46:40,185 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 2-902104676: Executing:  { Cmd , MgmtId: 95545481387, via: 2(x2.c.some.domain), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/45/372","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary","_role":"Image"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}] }
2014-12-08 08:46:40,186 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-205:ctx-eadb0ebb) Seq 2-902104676: Executing request
2014-12-08 08:46:40,757 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Creating a not shared SR for nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary/snapshots/45/372
2014-12-08 08:46:41,594 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Checking 57ffd43a-d997-395a-9be8-51d4d8922805 or SR de7d0b77-3559-8024-1dae-ac1ccdd088e0 on XS[99627967-f074-4312-8958-89f1745cce5b-XX.XX.XX.XX]
2014-12-08 08:46:41,606 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Created a SR; UUID is de7d0b77-3559-8024-1dae-ac1ccdd088e0 device config is {serverpath=/mnt/rootPool/nfsSecondary/snapshots/45/372, server=s6.c.some.domain}
2014-12-08 08:46:42,021 DEBUG [c.c.a.ApiServlet] (http-6443-exec-110:ctx-f9a8ba10) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024984486
2014-12-08 08:46:42,043 DEBUG [c.c.a.ApiServlet] (http-6443-exec-110:ctx-f9a8ba10 ctx-efef7156) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024984486
2014-12-08 08:46:42,493 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-1e3cd74d) Zone 1 is ready to launch console proxy
2014-12-08 08:46:42,581 DEBUG [c.c.s.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-29320eaf) Zone 1 is ready to launch secondary storage VM
2014-12-08 08:46:45,023 DEBUG [c.c.a.ApiServlet] (http-6443-exec-116:ctx-b1a42d14) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024987487
2014-12-08 08:46:45,053 DEBUG [c.c.a.ApiServlet] (http-6443-exec-116:ctx-b1a42d14 ctx-ca1e53d1) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024987487
2014-12-08 08:46:47,471 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-1:null) SeqA 27-899694: Processing Seq 27-899694:  { Cmd , MgmtId: -1, via: 27, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n  \"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:48,023 DEBUG [c.c.a.ApiServlet] (http-6443-exec-118:ctx-4fda25ac) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024990488
2014-12-08 08:46:49,820 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-1:null) SeqA 27-899694: Sending Seq 27-899694:  { Ans: , MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:49,820 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-05ac6324) Management server heartbeat takes too long to finish. profiler: Done. Duration: 2388ms, profilerHeartbeatUpdate: Done. Duration: 2385ms, profilerPeerScan: Done. Duration: 3ms
2014-12-08 08:46:49,828 DEBUG [c.c.a.ApiServlet] (http-6443-exec-118:ctx-4fda25ac ctx-42973d6d) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024990488
2014-12-08 08:46:51,023 DEBUG [c.c.a.ApiServlet] (http-6443-exec-120:ctx-ad2b5b3c) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024993488
2014-12-08 08:46:51,085 DEBUG [c.c.a.ApiServlet] (http-6443-exec-120:ctx-ad2b5b3c ctx-ef4952d2) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024993488
2014-12-08 08:46:54,024 DEBUG [c.c.a.ApiServlet] (http-6443-exec-121:ctx-d365a166) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024996489
2014-12-08 08:46:54,043 DEBUG [c.c.a.ApiServlet] (http-6443-exec-121:ctx-d365a166 ctx-3aa2a5d6) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024996489
2014-12-08 08:46:57,026 DEBUG [c.c.a.ApiServlet] (http-6443-exec-119:ctx-7cb8bab2) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024999490
2014-12-08 08:46:57,045 DEBUG [c.c.a.ApiServlet] (http-6443-exec-119:ctx-7cb8bab2 ctx-81dcf0d5) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024999490
2014-12-08 08:46:57,433 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) SeqA 27-899695: Processing Seq 27-899695:  { Cmd , MgmtId: -1, via: 27, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n  \"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:57,437 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) SeqA 27-899695: Sending Seq 27-899695:  { Ans: , MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:57,922 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
           nameLabel: Async.VDI.copy
     nameDescription: 
   allowedOperations: []
   currentOperations: {}
             created: Mon Dec 08 08:46:43 CET 2014
            finished: Mon Dec 08 08:46:57 CET 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@c612622b
            progress: 1.0
                type: <none/>
              result: 
           errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

2014-12-08 08:46:57,937 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Removing SR
2014-12-08 08:46:57,948 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX OpaqueRef:56da730b-b5dc-f972-6a8e-74ee643ac7a8: Unplugging pbd
2014-12-08 08:47:00,026 DEBUG [c.c.a.ApiServlet] (http-6443-exec-110:ctx-54e9aca1) ===START===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418025002491
2014-12-08 08:47:00,047 DEBUG [c.c.a.ApiServlet] (http-6443-exec-110:ctx-54e9aca1 ctx-619a48e5) ===END===  XX.XX.XX.XX -- GET  command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418025002491
2014-12-08 08:47:01,013 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ce02975d) Found 15 routers to update status. 
2014-12-08 08:47:01,016 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ce02975d) Found 0 networks to update RvR status. 
2014-12-08 08:47:01,586 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Forgetting
2014-12-08 08:47:01,604 WARN  [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-205:ctx-eadb0ebb) BackupSnapshot Failed due to Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
           nameLabel: Async.VDI.copy
     nameDescription: 
   allowedOperations: []
   currentOperations: {}
             created: Mon Dec 08 08:46:43 CET 2014
            finished: Mon Dec 08 08:46:57 CET 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@c612622b
            progress: 1.0
                type: <none/>
              result: 
           errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
           nameLabel: Async.VDI.copy
     nameDescription: 
   allowedOperations: []
   currentOperations: {}
             created: Mon Dec 08 08:46:43 CET 2014
            finished: Mon Dec 08 08:46:57 CET 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@c612622b
            progress: 1.0
                type: <none/>
              result: 
           errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

	at com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3764)
	at com.cloud.hypervisor.xen.resource.CitrixResourceBase.cloudVDIcopy(CitrixResourceBase.java:3942)
	at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1345)
	at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:90)
	at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:50)
	at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:609)
	at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
	at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:701)
2014-12-08 08:47:02,694 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-205:ctx-eadb0ebb) Seq 2-902104676: Response Received: 
2014-12-08 08:47:02,694 DEBUG [c.c.a.t.Request] (DirectAgent-205:ctx-eadb0ebb) Seq 2-902104676: Processing:  { Ans: , MgmtId: 95545481387, via: 2, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"BackupSnapshot Failed due to Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc\n           nameLabel: Async.VDI.copy\n     nameDescription: \n   allowedOperations: []\n   currentOperations: {}\n             created: Mon Dec 08 08:46:43 CET 2014\n            finished: Mon Dec 08 08:46:57 CET 2014\n              status: failure\n          residentOn: com.xensource.xenapi.Host@c612622b\n            progress: 1.0\n                type: <none/>\n              result: \n           errorInfo: [INTERNAL_ERROR, Failure(\"sparse_dd\")]\n         otherConfig: {}\n           subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n            subtasks: []\n","wait":0}}] }
2014-12-08 08:47:02,694 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Seq 2-902104676: Received:  { Ans: , MgmtId: 95545481387, via: 2, Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-12-08 08:47:02,724 DEBUG [c.c.s.s.SnapshotManagerImpl] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
           nameLabel: Async.VDI.copy
     nameDescription: 
   allowedOperations: []
   currentOperations: {}
             created: Mon Dec 08 08:46:43 CET 2014
            finished: Mon Dec 08 08:46:57 CET 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@c612622b
            progress: 1.0
                type: <none/>
              result: 
           errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

	at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:292)
	at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:142)
	at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:301)
	at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:945)
	at sun.reflect.GeneratedMethodAccessor357.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy160.takeSnapshot(Unknown Source)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1381)
	at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1783)
	at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1724)
	at sun.reflect.GeneratedMethodAccessor356.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy196.takeSnapshot(Unknown Source)
	at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
	at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
	at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:701)
2014-12-08 08:47:02,739 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Job-Executor-20:ctx-f5bed97c ctx-992184e6) Take snapshot: 372 failed
com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
	at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:963)
	at sun.reflect.GeneratedMethodAccessor357.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy160.takeSnapshot(Unknown Source)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1381)
	at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1783)
	at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1724)
	at sun.reflect.GeneratedMethodAccessor356.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy196.takeSnapshot(Unknown Source)
	at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
	at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
	at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:701)
Caused by: com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to Task failed! Task record:                 uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
           nameLabel: Async.VDI.copy
     nameDescription: 
   allowedOperations: []
   currentOperations: {}
             created: Mon Dec 08 08:46:43 CET 2014
            finished: Mon Dec 08 08:46:57 CET 2014
              status: failure
          residentOn: com.xensource.xenapi.Host@c612622b
            progress: 1.0
                type: <none/>
              result: 
           errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

	at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:292)
	at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:142)
	at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:301)
	at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:945)
	... 43 more
2014-12-08 08:47:02,741 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-20:ctx-f5bed97c) Complete async job-2834, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to create snapshot due to an internal error creating snapshot for volume 372"}
2014-12-08 08:47:02,746 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-20:ctx-f5bed97c) Done executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for job-2834
2014-12-08 08:47:02,750 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-20:ctx-f5bed97c) Remove job-2834 from job monitoring


Here is the log from X2 hypervisor, where VM resides:
Dec  8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 925/0 
Dec  8 08:46:36 x2 tapdisk[11369]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a: b: 51200, a: 6839, f: 981, n: 28075704 
Dec  8 08:46:36 x2 tapdisk[11369]: closed image /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a (0 users, state: 0x00000000, type: 4) 
Dec  8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:46:36 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69: b: 51200, a: 7758, f: 1809, n: 0 
Dec  8 08:46:36 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:46:36 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc: b: 30720, a: 27801, f: 26067, n: 0 
Dec  8 08:46:36 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:46:36 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e: b: 30720, a: 5845, f: 4996, n: 0 
Dec  8 08:46:36 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:37 x2 multipathd: dm-49: remove map (uevent) 
Dec  8 08:46:37 x2 multipathd: dm-49: devmap not registered, can't remove 
Dec  8 08:46:37 x2 multipathd: dm-49: remove map (uevent) 
Dec  8 08:46:37 x2 multipathd: dm-49: devmap not registered, can't remove 
Dec  8 08:46:38 x2 multipathd: dm-4: add map (uevent) 
Dec  8 08:46:38 x2 multipathd: dm-4: add map (uevent) 
Dec  8 08:46:38 x2 multipathd: dm-49: add map (uevent) 
Dec  8 08:46:38 x2 multipathd: dm-49: add map (uevent) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 8455 
Dec  8 08:46:38 x2 tapdisk[11369]: opened image /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a (1 users, state: 0x00000001, type: 4, rw) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 version: tap 0x00010003, b: 51200, a: 6839, f: 981, n: 0 
Dec  8 08:46:38 x2 tapdisk[11369]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 version: tap 0x00010003, b: 51200, a: 7758, f: 1809, n: 0 
Dec  8 08:46:38 x2 tapdisk[11369]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc version: tap 0x00010003, b: 30720, a: 27801, f: 26067, n: 0 
Dec  8 08:46:38 x2 tapdisk[11369]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e version: tap 0x00010003, b: 30720, a: 5845, f: 4996, n: 0 
Dec  8 08:46:38 x2 tapdisk[11369]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:38 x2 tapdisk[11369]: VBD CHAIN: 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:38 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 multipathd: dm-59: add map (uevent) 
Dec  8 08:46:47 x2 tapdisk[2030]: tapdisk-control: init, 10 x 4k buffers 
Dec  8 08:46:47 x2 tapdisk[2030]: I/O queue driver: lio 
Dec  8 08:46:47 x2 tapdisk[2030]: tapdisk-log: started, level 0 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9 version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 0 
Dec  8 08:46:47 x2 tapdisk[2030]: opened image /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9 (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 version: tap 0x00010003, b: 51200, a: 6839, f: 981, n: 0 
Dec  8 08:46:47 x2 tapdisk[2030]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 version: tap 0x00010003, b: 51200, a: 7758, f: 1809, n: 0 
Dec  8 08:46:47 x2 tapdisk[2030]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc version: tap 0x00010003, b: 30720, a: 27801, f: 26067, n: 0 
Dec  8 08:46:47 x2 tapdisk[2030]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e version: tap 0x00010003, b: 30720, a: 5845, f: 4996, n: 0 
Dec  8 08:46:47 x2 tapdisk[2030]: opened image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e (1 users, state: 0x00000003, type: 4, ro) 
Dec  8 08:46:47 x2 tapdisk[2030]: VBD CHAIN: 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e: type:vhd(4) storage:lvm(3) 
Dec  8 08:46:47 x2 tapdisk[2030]: bdev: capacity=209715200 sector_size=512/512 flags=0
Dec  8 08:46:47 x2 multipathd: tdi: add path (uevent) 
Dec  8 08:46:47 x2 multipathd: tdi: failed to store path info 
Dec  8 08:46:47 x2 multipathd: uevent trigger error 
Dec  8 08:46:47 x2 kernel: block tdi: sector-size: 512/512 capacity: 209715200
Dec  8 08:46:47 x2 kernel: blkback: event-channel 79
Dec  8 08:46:47 x2 kernel: blkback: ring-ref 251
Dec  8 08:46:47 x2 kernel: blkback: protocol 1 (x86_32-abi)
Dec  8 08:46:47 x2 kernel: blkfront: xvda: barriers enabled
Dec  8 08:46:47 x2 multipathd: xvda: add path (uevent) 
Dec  8 08:46:47 x2 multipathd: xvda: failed to store path info 
Dec  8 08:46:47 x2 multipathd: uevent trigger error 
Dec  8 08:46:47 x2 kernel:  xvda: xvda1 xvda2
Dec  8 08:46:48 x2 vbd.uevent[add](backend/vbd/0/51712): wrote /xapi/0/hotplug/vbd/51712/hotplug = 'online'
Dec  8 08:46:48 x2 vbd.uevent[add](backend/vbd/0/51712): wrote backend/vbd/0/51712/hotplug-status = 'connected'
Dec  8 08:46:50 x2 tapdisk[2373]: tapdisk-control: init, 10 x 4k buffers 
Dec  8 08:46:50 x2 tapdisk[2373]: I/O queue driver: lio 
Dec  8 08:46:50 x2 tapdisk[2373]: tapdisk-log: started, level 0 
Dec  8 08:46:50 x2 tapdisk[2373]: /var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 417 
Dec  8 08:46:50 x2 tapdisk[2373]: opened image /var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd (1 users, state: 0x00000001, type: 4, rw) 
Dec  8 08:46:50 x2 tapdisk[2373]: VBD CHAIN: 
Dec  8 08:46:50 x2 tapdisk[2373]: /var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd: type:vhd(4) storage:nfs(1) 
Dec  8 08:46:50 x2 tapdisk[2373]: bdev: capacity=209715200 sector_size=512/512 flags=0
Dec  8 08:46:50 x2 multipathd: tdo: add path (uevent) 
Dec  8 08:46:50 x2 multipathd: tdo: failed to store path info 
Dec  8 08:46:50 x2 multipathd: uevent trigger error 
Dec  8 08:46:50 x2 kernel: block tdo: sector-size: 512/512 capacity: 209715200
Dec  8 08:46:50 x2 kernel: blkback: event-channel 81
Dec  8 08:46:50 x2 kernel: blkback: ring-ref 130
Dec  8 08:46:50 x2 kernel: blkback: protocol 1 (x86_32-abi)
Dec  8 08:46:50 x2 multipathd: xvdb: add path (uevent) 
Dec  8 08:46:50 x2 multipathd: xvdb: failed to store path info 
Dec  8 08:46:50 x2 multipathd: uevent trigger error 
Dec  8 08:46:50 x2 kernel: blkfront: xvdb: barriers enabled
Dec  8 08:46:50 x2 kernel:  xvdb: unknown partition table
Dec  8 08:46:50 x2 vbd.uevent[add](backend/vbd/0/51728): wrote /xapi/0/hotplug/vbd/51728/hotplug = 'online'
Dec  8 08:46:50 x2 vbd.uevent[add](backend/vbd/0/51728): wrote backend/vbd/0/51728/hotplug-status = 'connected'
Dec  8 08:46:51 x2 sparse_dd: [ info|x2.c.some.domain|0||sparse_dd] streaming from raw /dev/xvda using BAT from /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9 (relative to None) to raw file:///dev/xvdb
Dec  8 08:46:51 x2 fe: 2421 (/opt/xensource/libexec/sparse_dd -machine -src /dev/xvda -dest /dev/xvdb -siz...) exitted with code 2
Dec  8 08:46:51 x2 xapi: [error|x2.c.some.domain|1285367 inet-RPC|Async.VDI.copy R:dd071aa37b78|xapi] sparse_dd exit: 2 
Dec  8 08:46:51 x2 xapi: [error|x2.c.some.domain|1285367 inet-RPC|Async.VDI.copy R:dd071aa37b78|xapi] Failure from sparse_dd: Fatal error: exception Invalid_argument("index out of bounds")  
Dec  8 08:46:51 x2 multipathd: xvdb: remove path (uevent) 
Dec  8 08:46:51 x2 multipathd: xvdb: spurious uevent, path not in pathvec 
Dec  8 08:46:51 x2 multipathd: uevent trigger error 
Dec  8 08:46:51 x2 vbd.uevent[remove](backend/vbd/0/51728): removed /xapi/0/hotplug/vbd/51728/hotplug
Dec  8 08:46:51 x2 vbd.uevent[remove](backend/vbd/0/51728): removed backend/vbd/0/51728/hotplug-status
Dec  8 08:46:52 x2 multipathd: tdo: remove path (uevent) 
Dec  8 08:46:52 x2 multipathd: tdo: spurious uevent, path not in pathvec 
Dec  8 08:46:52 x2 multipathd: uevent trigger error 
Dec  8 08:46:52 x2 tapdisk[2373]: gaps written/skipped: 0/0 
Dec  8 08:46:52 x2 tapdisk[2373]: /var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd: b: 51200, a: 0, f: 0, n: 417 
Dec  8 08:46:52 x2 tapdisk[2373]: closed image /var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd (0 users, state: 0x00000000, type: 4) 
Dec  8 08:46:52 x2 tapdisk[2373]: tapdisk-log: closing after 0 errors 
Dec  8 08:46:52 x2 tapdisk[2373]: tapdisk-syslog: 11 messages, 902 bytes, xmits: 12, failed: 0, dropped: 0
Dec  8 08:46:52 x2 tapdisk[2373]: tapdisk-control: draining 1 connections 
Dec  8 08:46:52 x2 tapdisk[2373]: tapdisk-control: done 
Dec  8 08:46:53 x2 multipathd: xvda: remove path (uevent) 
Dec  8 08:46:53 x2 multipathd: xvda: spurious uevent, path not in pathvec 
Dec  8 08:46:53 x2 multipathd: uevent trigger error 
Dec  8 08:46:53 x2 vbd.uevent[remove](backend/vbd/0/51712): removed /xapi/0/hotplug/vbd/51712/hotplug
Dec  8 08:46:53 x2 vbd.uevent[remove](backend/vbd/0/51712): removed backend/vbd/0/51712/hotplug-status
Dec  8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0 
Dec  8 08:46:54 x2 multipathd: tdi: remove path (uevent) 
Dec  8 08:46:54 x2 multipathd: tdi: spurious uevent, path not in pathvec 
Dec  8 08:46:54 x2 multipathd: uevent trigger error 
Dec  8 08:46:54 x2 tapdisk[2030]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9: b: 51200, a: 0, f: 0, n: 0 
Dec  8 08:46:54 x2 tapdisk[2030]: closed image /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0 
Dec  8 08:46:54 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959: b: 51200, a: 6839, f: 981, n: 0 
Dec  8 08:46:54 x2 tapdisk[2030]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0 
Dec  8 08:46:54 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69: b: 51200, a: 7758, f: 1809, n: 0 
Dec  8 08:46:54 x2 tapdisk[2030]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0 
Dec  8 08:46:54 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc: b: 30720, a: 27801, f: 26067, n: 0 
Dec  8 08:46:54 x2 tapdisk[2030]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0 
Dec  8 08:46:54 x2 tapdisk[2030]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e: b: 30720, a: 5845, f: 4996, n: 0 
Dec  8 08:46:54 x2 tapdisk[2030]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e (0 users, state: 0x00000002, type: 4) 
Dec  8 08:46:54 x2 tapdisk[2030]: tapdisk-log: closing after 0 errors 
Dec  8 08:46:54 x2 tapdisk[2030]: tapdisk-syslog: 15 messages, 1603 bytes, xmits: 17, failed: 0, dropped: 0
Dec  8 08:46:54 x2 tapdisk[2030]: tapdisk-control: draining 1 connections 
Dec  8 08:46:54 x2 tapdisk[2030]: tapdisk-control: done 
Dec  8 08:46:55 x2 multipathd: dm-59: remove map (uevent) 
Dec  8 08:46:55 x2 multipathd: dm-59: devmap not registered, can't remove 
Dec  8 08:46:55 x2 multipathd: dm-59: remove map (uevent) 
Dec  8 08:46:55 x2 multipathd: dm-59: devmap not registered, can't remove 
Dec  8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:47:07 x2 tapdisk[11369]: /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a: b: 51200, a: 54, f: 0, n: 230064 
Dec  8 08:47:07 x2 tapdisk[11369]: closed image /dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a (0 users, state: 0x00000000, type: 4) 
Dec  8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:47:07 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959: b: 51200, a: 6839, f: 981, n: 0 
Dec  8 08:47:07 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:47:07 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69: b: 51200, a: 7758, f: 1809, n: 0 
Dec  8 08:47:07 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69 (0 users, state: 0x00000002, type: 4) 
Dec  8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:47:07 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc: b: 30720, a: 27801, f: 26067, n: 0 
Dec  8 08:47:07 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc (0 users, state: 0x00000002, type: 4) 
Dec  8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0 
Dec  8 08:47:07 x2 tapdisk[11369]: /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e: b: 30720, a: 5845, f: 4996, n: 0 
Dec  8 08:47:07 x2 tapdisk[11369]: closed image /dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e (0 users, state: 0x00000002, type: 4) 
Dec  8 08:47:09 x2 multipathd: dm-4: remove map (uevent) 
Dec  8 08:47:09 x2 multipathd: dm-4: devmap not registered, can't remove 
Dec  8 08:47:09 x2 multipathd: dm-4: remove map (uevent) 
Dec  8 08:47:09 x2 multipathd: dm-4: devmap not registered, can't remove 
Dec  8 08:47:09 x2 multipathd: dm-4: add map (uevent) 
Dec  8 08:47:09 x2 multipathd: dm-4: add map (uevent) 
Dec  8 08:47:09 x2 multipathd: dm-59: add map (uevent) 
Dec  8 08:47:09 x2 multipathd: dm-59: add map (uevent) 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message