cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Likitha Shetty <likitha.she...@citrix.com>
Subject RE: "Bug" introduced by part of commit a600d8408ea86782318139c17cf346c8497943d0
Date Thu, 10 Jul 2014 08:49:38 GMT
Funs, Santhosh fixed this problem yesterday. https://issues.apache.org/jira/browse/CLOUDSTACK-7079
has more details.

Thanks,
Likitha

-----Original Message-----
From: Funs Kessen [mailto:FKessen@schubergphilis.com] 
Sent: Thursday, July 10, 2014 2:10 PM
To: dev@cloudstack.apache.org
Cc: Daan Hoogland
Subject: "Bug" introduced by part of commit a600d8408ea86782318139c17cf346c8497943d0

Hi Devs,

I recently noticed on master that after starting, stopping and starting the management server
again I get stacktraces about every second.
After some digging with Daan we found that part of the a600d8408ea86782318139c17cf346c84979943d0
commit causes this. The problem is however deeper rooted, as the code is supposed to update
the mshost table, but because another Session ID (actually _runId from ClusterManagerImpl.java
from the previous run)  is already in there the update fails. The Session ID is based on time
in milliseconds, and offcourse changes when you stop and start the management server again.
Prior to the commit it failed silently, and has done so since the initial checkin it seems.
The real question is what the original idea is behind the _runId, and is it something that
requires fixing ?

I've pasted part of the commit, sql and stacktrace below.

Cheers,

Funs

===
mysql> select * from mshost;
+----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
| id | msid           | runid         | name    | state | version        | service_ip | service_port
| last_update         | removed | alert_count |
+----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
|  1 | 90520734207775 | 1404924979461 | cs-mgmt | Up    | 4.5.0-SNAPSHOT | 127.0.0.1  |  
      9090 | 2014-07-09 16:58:07 | NULL    |           0 |
+----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
1 row in set (0.00 sec)

===
commit a600d8408ea86782318139c17cf346c8497943d0
Author: Santhosh Edukulla <santhosh.edukulla@gmail.com> 2014-07-02 10:38:16
Committer: Santhosh Edukulla <santhosh.edukulla@gmail.com> 2014-07-04 12:47:58

Fixed Resource Leaks, null dereferences, few other issues reported by coverity

- framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java 
diff --git a/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
b/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
index 3d0c3f5..89d7d27 100644
--- a/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
+++ b/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
@@ -53,15 +53,14 @@
     @Override
     public void invalidateRunSession(long id, long runid) {
         TransactionLegacy txn = TransactionLegacy.currentTxn();
-        PreparedStatement pstmt = null;
-        try {
-            pstmt = txn.prepareAutoCloseStatement("update mshost set runid=0, state='Down'
where id=? and runid=?");
-            pstmt.setLong(1, id);
-            pstmt.setLong(2, runid);
-
-            pstmt.executeUpdate();
+        try (PreparedStatement pstmt = txn.prepareStatement("update mshost set runid=0, state='Down'
where id=? and runid=?");){
+            if(pstmt != null) {
+                pstmt.setLong(1, id);
+                pstmt.setLong(2, runid);
+                pstmt.executeUpdate();
+            }
         } catch (SQLException e) {
-            throw new CloudRuntimeException("DB exception on " + pstmt.toString(), e);
+            throw new CloudRuntimeException("invalidateRunSession:Exception:"+ e.getMessage(),
e);
         }
     }

===
2014-07-09 18:45:06,541 WARN  [c.c.c.d.ManagementServerHostDaoImpl] (Cluster-Heartbeat-1:ctx-5f2f8ad5)
update:Exception:Invalid cluster session detected
com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected
        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy158.update(Unknown Source)
        at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404924278923 is no longer
valid
        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
        ... 26 more
2014-07-09 18:45:06,543 ERROR [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-5f2f8ad5)
Unexpected exception in cluster heartbeat
java.lang.RuntimeException: update:Exception:Invalid cluster session detected
        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:155)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy158.update(Unknown Source)
        at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected
        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
        ... 26 more
Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404924278923 is no longer
valid
        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
        ... 26 more




Mime
View raw message