cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hugo Trippaers <h...@trippaers.nl>
Subject Re: "Bug" introduced by part of commit a600d8408ea86782318139c17cf346c8497943d0
Date Thu, 10 Jul 2014 08:58:10 GMT
Doesn’t seem related to that particular commit, the function invalidateRunSession appears
to be dead code.

I’ll do some digging.


Cheers,

Hugo

On 10 jul. 2014, at 10:40, Funs Kessen <FKessen@schubergphilis.com> wrote:

> Hi Devs,
> 
> I recently noticed on master that after starting, stopping and starting the management
server again I get stacktraces about every second.
> After some digging with Daan we found that part of the a600d8408ea86782318139c17cf346c84979943d0
commit causes this. The problem is however deeper rooted, as the code is supposed to update
the mshost table, but because another Session ID (actually _runId from ClusterManagerImpl.java
from the previous run)  is already in there the update fails. The Session ID is based on time
in milliseconds, and offcourse changes when you stop and start the management server again.
> Prior to the commit it failed silently, and has done so since the initial checkin it
seems. The real question is what the original idea is behind the _runId, and is it something
that requires fixing ?
> 
> I've pasted part of the commit, sql and stacktrace below.
> 
> Cheers,
> 
> Funs
> 
> ===
> mysql> select * from mshost;
> +----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
> | id | msid           | runid         | name    | state | version        | service_ip
| service_port | last_update         | removed | alert_count |
> +----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
> |  1 | 90520734207775 | 1404924979461 | cs-mgmt | Up    | 4.5.0-SNAPSHOT | 127.0.0.1
 |         9090 | 2014-07-09 16:58:07 | NULL    |           0 |
> +----+----------------+---------------+---------+-------+----------------+------------+--------------+---------------------+---------+-------------+
> 1 row in set (0.00 sec)
> 
> ===
> commit a600d8408ea86782318139c17cf346c8497943d0
> Author: Santhosh Edukulla <santhosh.edukulla@gmail.com> 2014-07-02 10:38:16
> Committer: Santhosh Edukulla <santhosh.edukulla@gmail.com> 2014-07-04 12:47:58
> 
> Fixed Resource Leaks, null dereferences, few other issues reported by coverity
> 
> - framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java 
> diff --git a/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
b/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
> index 3d0c3f5..89d7d27 100644
> --- a/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
> +++ b/framework/cluster/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
> @@ -53,15 +53,14 @@
>     @Override
>     public void invalidateRunSession(long id, long runid) {
>         TransactionLegacy txn = TransactionLegacy.currentTxn();
> -        PreparedStatement pstmt = null;
> -        try {
> -            pstmt = txn.prepareAutoCloseStatement("update mshost set runid=0, state='Down'
where id=? and runid=?");
> -            pstmt.setLong(1, id);
> -            pstmt.setLong(2, runid);
> -
> -            pstmt.executeUpdate();
> +        try (PreparedStatement pstmt = txn.prepareStatement("update mshost set runid=0,
state='Down' where id=? and runid=?");){
> +            if(pstmt != null) {
> +                pstmt.setLong(1, id);
> +                pstmt.setLong(2, runid);
> +                pstmt.executeUpdate();
> +            }
>         } catch (SQLException e) {
> -            throw new CloudRuntimeException("DB exception on " + pstmt.toString(), e);
> +            throw new CloudRuntimeException("invalidateRunSession:Exception:"+ e.getMessage(),
e);
>         }
>     }
> 
> ===
> 2014-07-09 18:45:06,541 WARN  [c.c.c.d.ManagementServerHostDaoImpl] (Cluster-Heartbeat-1:ctx-5f2f8ad5)
update:Exception:Invalid cluster session detected
> com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected
>        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:606)
>        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>        at com.sun.proxy.$Proxy158.update(Unknown Source)
>        at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545)
>        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>        at java.lang.Thread.run(Thread.java:744)
> Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404924278923 is no
longer valid
>        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
>        ... 26 more
> 2014-07-09 18:45:06,543 ERROR [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-5f2f8ad5)
Unexpected exception in cluster heartbeat
> java.lang.RuntimeException: update:Exception:Invalid cluster session detected
>        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:155)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:606)
>        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>        at com.sun.proxy.$Proxy158.update(Unknown Source)
>        at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545)
>        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>        at java.lang.Thread.run(Thread.java:744)
> Caused by: com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected
>        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
>        ... 26 more
> Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404924278923 is no
longer valid
>        at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147)
>        ... 26 more
> 
> 
> 


Mime
View raw message