cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Min Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-4625) Snapshots and templates should be deleted from staging storage after create template from snapshot on S3.
Date Fri, 06 Sep 2013 21:43:53 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Min Chen updated CLOUDSTACK-4625:
---------------------------------

    Description: 
storage artifacts like snapshots and templates are not getting deleted from staging storage
when template was created from snapshot.

Following actions take place when template is created from snapshot:
1.Snapshot gets downloaded from S3 to snapshots directory in NFS staging storage
2.Then it gets copied to template directory in NFS staging storage
3.Finally it is uploaded to S3
After successful operation snapshot and template copied to staging storage (in step1 and step2)
should be deleted.


  was:
volume copied to secondary staging storage during volume migration should be deleted after
migration.

Steps to Reproduce:
=================
1.Bring up CS with xen cluster using S3 from image store, NFS for secondary staging and primary
storage
2.Deploy guest vm using default cent os template with both Root and Data disks.
3.Add another NFS based primary storage to the cluster 
4.Detach data disk from the vm
5.Migrate the data disk to the primary storage created at step3

Result:
======
Volume migration was successful. But the volume copied to secondary staging storage during
migration process did not get deleted.
Only volume from source primary storage got deleted.

Observations:
============
Following is the log snippet during volume migration:
2013-08-08 08:34:49,766 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===START=== 
10.146.0.20 -- GET  command=migrateVolume&storageid=29a0c990-7100-3a8d-b570-ba9f84ca78bc&volumeid=e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef&response=json&sessionkey=qXfe5TLEOA5koD0qobFirCKKbOY%3D&_=1375965252817
2013-08-08 08:34:49,954 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-13:null) submit
async job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ], details: AsyncJobVO {id:44, userId:
2, accountId: 2, sessionKey: null, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd,
cmdOriginator: null, cmdInfo: {"response":"json","sessionkey":"qXfe5TLEOA5koD0qobFirCKKbOY\u003d","cmdEventType":"VOLUME.MIGRATE","ctxUserId":"2","storageid":"29a0c990-7100-3a8d-b570-ba9f84ca78bc","httpmethod":"GET","volumeid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","_":"1375965252817","ctxAccountId":"2","ctxStartEventId":"194"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode:
0, result: null, initMsid: 6615759585382, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2013-08-08 08:34:49,957 DEBUG [cloud.api.ApiServlet] (catalina-exec-13:null) ===END===  10.146.0.20
-- GET  command=migrateVolume&storageid=29a0c990-7100-3a8d-b570-ba9f84ca78bc&volumeid=e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef&response=json&sessionkey=qXfe5TLEOA5koD0qobFirCKKbOY%3D&_=1375965252817
2013-08-08 08:34:49,961 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-45:job-44 =
[ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]) Executing org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd
for job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]
2013-08-08 08:34:50,032 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-45:job-44
= [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]) copyAsync inspecting src type VOLUME copyAsync
inspecting dest type VOLUME
2013-08-08 08:34:50,061 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-45:job-44
= [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]) copyAsync inspecting src type VOLUME copyAsync
inspecting dest type VOLUME
2013-08-08 08:34:50,083 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303576995: Sending  { Cmd , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":3,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"cs-3416","size":5368709120,"path":"5a436a4d-5758-4699-b4d1-7276e1ceb4e5","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_xen_os","_role":"ImageCache"}},"name":"cs-3416","size":5368709120,"path":"volumes/2/14","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"executeInSequence":false,"wait":10800}}]
}
2013-08-08 08:34:50,085 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303576995: Executing:  { Cmd , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":3,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"cs-3416","size":5368709120,"path":"5a436a4d-5758-4699-b4d1-7276e1ceb4e5","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_xen_os","_role":"ImageCache"}},"name":"cs-3416","size":5368709120,"path":"volumes/2/14","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"executeInSequence":false,"wait":10800}}]
}
2013-08-08 08:35:20,118 DEBUG [cloud.api.ApiServlet] (catalina-exec-10:null) ===START=== 
10.146.0.20 -- GET  command=queryAsyncJobResult&jobId=3c1fd226-af63-47fe-9a5c-fc4770f1a6f5&response=json&sessionkey=qXfe5TLEOA5koD0qobFirCKKbOY%3D&_=1375965283221
2013-08-08 08:35:20,246 DEBUG [cloud.api.ApiServlet] (catalina-exec-10:null) ===END===  10.146.0.20
-- GET  command=queryAsyncJobResult&jobId=3c1fd226-af63-47fe-9a5c-fc4770f1a6f5&response=json&sessionkey=qXfe5TLEOA5koD0qobFirCKKbOY%3D&_=1375965283221
2013-08-08 08:35:20,676 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-79:null) Seq
2-1303576999: Response Received:
2013-08-08 08:35:20,676 DEBUG [agent.transport.Request] (DirectAgent-79:null) Seq 2-1303576999:
Processing:  { Ans: , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"size":5368709120,"path":"5a6f328e-cb37-447d-90e9-eb3efaa4e0db","accountId":0,"id":0}},"result":true,"wait":0}}]
}
2013-08-08 08:35:20,677 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303576999: Received:  { Ans: , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 10,
{ CopyCmdAnswer } }
2013-08-08 08:35:20,778 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303577000: Sending  { Cmd , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":3,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"cs-3416","size":5368709120,"path":"5a436a4d-5758-4699-b4d1-7276e1ceb4e5","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"wait":0}}]
}
2013-08-08 08:35:20,779 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303577000: Executing:  { Cmd , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e1eb0b93-3fba-4437-ad50-b12bf1d6f1ef","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":3,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"cs-3416","size":5368709120,"path":"5a436a4d-5758-4699-b4d1-7276e1ceb4e5","volumeId":14,"accountId":2,"format":"VHD","id":14,"hypervisorType":"XenServer"}},"wait":0}}]
}
2013-08-08 08:35:20,780 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-90:null) Seq
2-1303577000: Executing request
2013-08-08 08:35:21,440 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-90:null) Seq
2-1303577000: Response Received:
2013-08-08 08:35:21,441 DEBUG [agent.transport.Request] (DirectAgent-90:null) Seq 2-1303577000:
Processing:  { Ans: , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
2013-08-08 08:35:21,441 DEBUG [agent.transport.Request] (Job-Executor-45:job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
]) Seq 2-1303577000: Received:  { Ans: , MgmtId: 6615759585382, via: 2, Ver: v1, Flags: 10,
{ Answer } }
2013-08-08 08:35:21,562 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-45:job-44 =
[ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]) Complete async job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5
], jobStatus: 1, resultCode: 0, result: org.apache.cloudstack.api.response.VolumeResponse@388699b5
2013-08-08 08:35:21,578 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-45:job-44 =
[ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]) Done executing org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd
for job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ]
2013-08-08 08:35:23,171 DEBUG [cloud.api.ApiServlet] (catalina-exec-14:null) ===START=== 
10.146.0.20 -- GET  command=queryAsyncJobResult&jobId=3c1fd226-af63-47fe-9a5c-fc4770f1a6f5&response=json&sessionkey=qXfe5TLEOA5koD0qobFirCKKbOY%3D&_=1375965286243
2013-08-08 08:35:23,199 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-14:null) Async
job-44 = [ 3c1fd226-af63-47fe-9a5c-fc4770f1a6f5 ] completed

During volume migration process, an entry will be made in volume_store_ref table with install_path
points to the volume location in the secondary staging storage.

Attaching management server log file and cloud DB.


       Assignee:     (was: Min Chen)
    
> Snapshots and templates should be deleted from staging storage after create template
from snapshot on S3.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4625
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4625
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Storage Controller, Volumes
>    Affects Versions: 4.2.0
>         Environment: Latest build from ACS 4.2 branch
> Storage: S3 for image store, NFS for secondary staging and primary storage
>            Reporter: Min Chen
>            Priority: Critical
>             Fix For: 4.2.1
>
>
> storage artifacts like snapshots and templates are not getting deleted from staging storage
when template was created from snapshot.
> Following actions take place when template is created from snapshot:
> 1.Snapshot gets downloaded from S3 to snapshots directory in NFS staging storage
> 2.Then it gets copied to template directory in NFS staging storage
> 3.Finally it is uploaded to S3
> After successful operation snapshot and template copied to staging storage (in step1
and step2) should be deleted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message