cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hiroki Ohashi <hiroki.s...@gmail.com>
Subject CLOUDSTACK-6236 isn't completely fixed?
Date Tue, 24 Jun 2014 05:57:32 GMT
Dear all

I have a production environment of CloudStack 4.3.0 build from
4.3.0 branch source code and encountered an exception like [1]. I
suspect that this is caused by a bug of CLOUDSTACK-6236 because a
patch isn't applied to CloudStack 4.3.0 branch.

(1) If my suspect is correct, a database is probably in inconsistent
state and need to be repaired. Then, would you tell me how to fix the
database if you know it?

(2) Please cherry-pick a patch of CLOUDSTACK-6236 to 4.3.0 branch.

(3) setRefCnt method is modified in VolumeDataStoreVO.java by the
patch of CLOUDSTACK-6236 but is this a mistake of decrRefCnt method
below it in [2]? If the assumption is right, the bug fix is
incomplete. Would you please review a patch that fixes the issue
written by my co-worker below [3]?


Best Regards

[1]: exception example

2014-06-23 17:29:46,384 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Job-Executor-57:ctx-d0af85f8 ctx-4b2c5b2b) copy object failed:
com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
com.mysql.jdbc.JDBC4PreparedStatement@64be8bcb: SELECT
template_store_ref.id, template_store_ref.store_id,
template_store_ref.template_id, template_store_ref.store_role,
template_store_ref.created, template_store_ref.last_updated,
template_store_ref.download_pct, template_store_ref.size,
template_store_ref.physical_size, template_store_ref.download_state,
template_store_ref.local_path, template_store_ref.error_str,
template_store_ref.job_id, template_store_ref.install_path,
template_store_ref.url, template_store_ref.is_copy,
template_store_ref.destroyed, template_store_ref.update_count,
template_store_ref.updated, template_store_ref.state,
template_store_ref.ref_cnt FROM template_store_ref WHERE
template_store_ref.store_id = 1  AND template_store_ref.template_id =
245  AND template_store_ref.destroyed = 0  ORDER BY RAND() LIMIT 1
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:421)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:356)
        at com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:869)
        at org.apache.cloudstack.storage.image.db.TemplateDataStoreDaoImpl.findByStoreTemplate(TemplateDataStoreDaoImpl.java:320)
        at sun.reflect.GeneratedMethodAccessor124.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:33)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy109.findByStoreTemplate(Unknown Source)
        at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.findObject(ObjectInDataStoreManagerImpl.java:347)
        at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.findObject(ObjectInDataStoreManagerImpl.java:338)
        at org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.createCacheObject(StorageCacheManagerImpl.java:219)
        at org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.createCacheObject(StorageCacheManagerImpl.java:267)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:165)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:428)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:70)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:473)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:604)
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1192)
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1256)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:962)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:761)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:601)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:207)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3554)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3161)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3147)
        at sun.reflect.GeneratedMethodAccessor499.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy170.startVirtualMachine(Unknown Source)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:443)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
        at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
        at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:701)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLDataException:
'18446744073709551615' in column '21' is outside valid range for the
datatype BIGINT.
        at sun.reflect.GeneratedConstructorAccessor468.newInstance(Unknown
Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        at com.mysql.jdbc.Util.getInstance(Util.java:386)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1026)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
        at com.mysql.jdbc.ResultSetImpl.throwRangeException(ResultSetImpl.java:7964)
        at com.mysql.jdbc.ResultSetImpl.parseLongAsDouble(ResultSetImpl.java:7248)
        at com.mysql.jdbc.ResultSetImpl.getLong(ResultSetImpl.java:2946)
        at com.mysql.jdbc.ResultSetImpl.getLong(ResultSetImpl.java:2911)
        at org.apache.commons.dbcp.DelegatingResultSet.getLong(DelegatingResultSet.java:228)
        at org.apache.commons.dbcp.DelegatingResultSet.getLong(DelegatingResultSet.java:228)
        at com.cloud.utils.db.GenericDaoBase.setField(GenericDaoBase.java:541)
        at com.cloud.utils.db.GenericDaoBase.setField(GenericDaoBase.java:1702)
        at com.cloud.utils.db.GenericDaoBase.toEntityBean(GenericDaoBase.java:1582)
        at com.cloud.utils.db.GenericDaoBase.toEntityBean(GenericDaoBase.java:1543)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:417)
        ... 66 more

[2]: <master> engine/schema/src/org/apache/cloudstack/storage/datastore/db/VolumeDataStoreVO.java

    352     public void setRefCnt(Long refCnt) {
    353         if (refCnt > 0) {
    354             refCnt--;
    355         }
    356         else {
    357             s_logger.warn("We should not try to decrement a
zero reference count even though our code has guarded");
    358         }
    359     }
    360
    361     public void incrRefCnt() {
    362         refCnt++;
    363     }
    364
    365     public void decrRefCnt() {
    366         refCnt--;
    367     }

[3]: https://reviews.apache.org/r/22925/

-- 
Hiroki Ohashi

Mime
View raw message