Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 796DA10652 for ; Fri, 11 Jul 2014 05:09:06 +0000 (UTC) Received: (qmail 16789 invoked by uid 500); 11 Jul 2014 05:09:06 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 16752 invoked by uid 500); 11 Jul 2014 05:09:05 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 16727 invoked by uid 500); 11 Jul 2014 05:09:05 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 16708 invoked by uid 99); 11 Jul 2014 05:09:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jul 2014 05:09:05 +0000 Date: Fri, 11 Jul 2014 05:09:05 +0000 (UTC) From: "sadhu suresh (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-7070) Continuous Error messages" Invalid cluster session detected" in MS logs after starting the management server MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058385#comment-14058385 ] sadhu suresh commented on CLOUDSTACK-7070: ------------------------------------------ I just updated the side effect of this bug and due to this bug no other action is triggered on MS. AS my execution blocked due to this bug and updated my comments while changing the priority by providing the impact of the bug. This bugs is not only filling the log and also impact other execution in my env. so I have updated the same bug. until and unless it got resolved I can't proceed further.so its better ito update the same bug. raising different bugs is not recommend if we blocked at initial stage it self. will test your fix and if I still see the same behaviour(i.e system vms are not coming up) then will raise separate bug by closing your bug if invalid cluster session detected got addressed. > Continuous Error messages" Invalid cluster session detected" in MS logs after starting the management server > ------------------------------------------------------------------------------------------------------------ > > Key: CLOUDSTACK-7070 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7070 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the default.) > Components: Management Server > Affects Versions: 4.5.0 > Reporter: manasaveloori > Assignee: Santhosh Kumar Edukulla > Priority: Blocker > Attachments: management-server.rar > > > Observed DB exception in logs which is mentioned in bug CLOUDSTACK-7041 > After this DB exception observing the follwoing continuous ERROR messages in logs: > 2014-07-07 15:14:40,043 WARN [c.c.c.d.ManagementServerHostDaoImpl] (Cluster-Heartbeat-1:ctx-49fa7a77) update:Exception:Invalid cluster session detected > com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected > at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) > at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) > at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) > at com.sun.proxy.$Proxy150.update(Unknown Source) > at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545) > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404760440284 is no longer valid > at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147) > ... 26 more > 2014-07-07 15:14:40,046 ERROR [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-49fa7a77) Unexpected exception in cluster heartbeat > java.lang.RuntimeException: update:Exception:Invalid cluster session detected > at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:155) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) > at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) > at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) > at com.sun.proxy.$Proxy150.update(Unknown Source) > at com.cloud.cluster.ClusterManagerImpl$4.runInContext(ClusterManagerImpl.java:545) > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > Caused by: com.cloud.utils.exception.CloudRuntimeException: Invalid cluster session detected > at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147) > ... 26 more > Caused by: com.cloud.cluster.ClusterInvalidSessionException: runid 1404760440284 is no longer valid > at com.cloud.cluster.dao.ManagementServerHostDaoImpl.update(ManagementServerHostDaoImpl.java:147) > ... 26 more > 2014-07-07 15:14:40,607 INFO [o.a.c.d.ApiDiscoveryServiceImpl] (main:null) Api Discovery Service: Annotation, docstrings, api relation graph processed in 1504.54192 ms > 2014-07-07 15:14:40,613 INFO [c.c.s.ManagementServerImpl] (main:null) Startup CloudStack management server... > 2014-07-07 15:14:40,616 INFO [c.c.c.ClusterManagerImpl] (main:null) register cluster listener class com.cloud.serve > Attaching the MSlogs -- This message was sent by Atlassian JIRA (v6.2#6252)