cloudstack-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [cloudstack] andrijapanicsb edited a comment on issue #3649: Garbage snapshots are left on Primary and Secondary
Date Fri, 22 Nov 2019 23:55:31 GMT
andrijapanicsb edited a comment on issue #3649: Garbage snapshots are left on Primary and Secondary
URL: https://github.com/apache/cloudstack/pull/3649#issuecomment-557715680
 
 
   Alright - so far tested only KVM (centos 7) with NFS:
   
   **While backup to secondary = true**, results as follows:
   - snap created on Prim Stor, copied to Sec Store, snap deleted from Prim Store (exists
only on Sec Store)
   - revert from snap is possible (image moved back and overwritten on Prim Store, as FULL
clone)
   - When deleting snap, it's instantly removed from Sec Stor NFS, DB records are changed
to "destroyed" state for both snap reference on Prim Store and on Sec Store but no DB records
removed so far.
   - CleanUp thread finds snaps in snapshos_store_ref, and removes them from the snapshots_store_ref
table - but it only removes the ones that are pointing to the Sec Store, not the ones that
are pointing to PrimStore (though they are marked as deleted, but rows physically present
in the table)
   
   In general, looking good, except that those records for snap reference on Prim Store are
not deleted from the snapshots_store_ref table - not sure what was the original behaviour
here, years ago.
   
   **While backup to secondary = false**, results as follows:
   - snap created and stays only on Prim Store, all good.
   - **can't delete the snap** - getting error _"Failed to delete snapshot:java.util.MissingFormatArgumentException:
Format specifier '%d'"_
   - cant revert volume from snap (seems to require to be on secondary storage, API confirms
this if you try to revert it via API) - what is the use of this setup then? I'm certainly
missing something or perhaps idea is that we have identical behaviour as many years ago, just
this time we don't copy it over to Sec Store @GabrielBrascher  (you can only create a template
or a volume out of it - can't restore it)
   
   Here is a more detailed error when deleting a snap that exists only on Prim Store:
   
   > 2019-11-22 22:18:42,983 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-12265ddd ctx-75ecde1c)
(logid:83e43335) ===END===  10.1.0.1 -- GET  command=deleteSnapshot&id=73120b09-10d7-48e0-bddb-42b442bbd79e&response=json&_=1574460913233
   > 2019-11-22 22:18:43,030 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-3:ctx-2cf618ce
job-75 ctx-39866784) (logid:6a5a4b68) Failed to delete snapshot: 7:java.util.MissingFormatArgumentException:
Format specifier '%d'
   > 2019-11-22 22:18:43,038 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-3:ctx-2cf618ce
job-75) (logid:6a5a4b68) Unexpected exception while executing org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd
   > com.cloud.utils.exception.CloudRuntimeException: Failed to delete snapshot:java.util.MissingFormatArgumentException:
Format specifier '%d'
   >         at com.cloud.storage.snapshot.SnapshotManagerImpl.deleteSnapshot(SnapshotManagerImpl.java:611)
   >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   >         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >         at java.lang.reflect.Method.invoke(Method.java:498)
   >         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
   >         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
   >         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   >         at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
   >         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
   >         at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
   >         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
   >         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
   >         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
   >         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
   >         at com.sun.proxy.$Proxy199.deleteSnapshot(Unknown Source)
   >         at org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd.execute(DeleteSnapshotCmd.java:103)
   >         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
   >         at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
   >         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:583)
   >         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
   >         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
   >         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
   >         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
   >         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
   >         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:531)
   >         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   >         at java.lang.Thread.run(Thread.java:748)
   > > 2019-11-22 22:18:43,035 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-1:ctx-3d310368
job-102) (logid:ed551d42) Complete async job-102, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to delete snapshot:java.util.MissingFormatArgumentException: Format specifier \u0027%d\u0027"}
   
   @GabrielBrascher I'm wondering, is the behaviour SAME in your tests with Ceph (you obviously
do NOT have issues deleting the snap from the Ceph only ( backup to Sec Store = false)
   Current behaviour that I see (snap removed from Prim when it's copied over to Sec) is the
new in this PR - i.e. previously snap was left on Prim Store even though it was copied over
to Sec Store.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message