Return-Path: X-Original-To: apmail-felix-dev-archive@www.apache.org Delivered-To: apmail-felix-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E33C0873 for ; Fri, 24 Aug 2012 14:09:25 +0000 (UTC) Received: (qmail 59035 invoked by uid 500); 24 Aug 2012 14:09:25 -0000 Delivered-To: apmail-felix-dev-archive@felix.apache.org Received: (qmail 58970 invoked by uid 500); 24 Aug 2012 14:09:24 -0000 Mailing-List: contact dev-help@felix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@felix.apache.org Delivered-To: mailing list dev@felix.apache.org Received: (qmail 58957 invoked by uid 99); 24 Aug 2012 14:09:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Aug 2012 14:09:24 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of pierre.derop@gmail.com designates 209.85.216.177 as permitted sender) Received: from [209.85.216.177] (HELO mail-qc0-f177.google.com) (209.85.216.177) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Aug 2012 14:09:08 +0000 Received: by qcsu28 with SMTP id u28so1543988qcs.22 for ; Fri, 24 Aug 2012 07:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bxQDQSkj3zNJ+nKh/3qitkiakjM0N953KmD1KWKkHOw=; b=lAlaztueXBFTsoI31YLhPj0HMWycXKT8PQWk0VjMCF3Rvll0HB5xhLgx3FuBLk6J5Q WU+qbVTM4rwPw103bSbmm14GEifkSZ64iA3ImBVGA8UbzvSZhH+YGH/PGyaNRZz9JMvW +i2PDvTlYyB1iZcTK3/E2Dqf3HvgzDmPIOLYI5dwuuL90WYO3xxGSM8VKDW3+g/XsyaH m1mVFiKMqTC4gYW0+RwAZFxKP7+JueZnBqZxnOLuWoShO/QrvApIsBpTjdPMj2xzab/T PCWdwixjsGYlZe59zptGiSgJGY6sta8S9Bu+ffkTIrG8cHIQw7rx1zSA3HwFA+WzVB3m B8rQ== MIME-Version: 1.0 Received: by 10.224.175.8 with SMTP id v8mr8897101qaz.66.1345817327549; Fri, 24 Aug 2012 07:08:47 -0700 (PDT) Received: by 10.49.120.2 with HTTP; Fri, 24 Aug 2012 07:08:47 -0700 (PDT) In-Reply-To: <078AC2F8-BC19-4675-8248-779C5D6C7F76@yahoo.com> References: <7B457C22-967F-486D-92FE-8D9FF59A3C61@yahoo.com> <078AC2F8-BC19-4675-8248-779C5D6C7F76@yahoo.com> Date: Fri, 24 Aug 2012 16:08:47 +0200 Message-ID: Subject: Re: SCR could not obtain state lock after 5 seconds From: Pierre De Rop To: dev@felix.apache.org Content-Type: multipart/alternative; boundary=485b397dd729bdbd0f04c8038328 --485b397dd729bdbd0f04c8038328 Content-Type: text/plain; charset=ISO-8859-1 Hi David, Sorry for my late response, I was off during the last two weeks. So, we finally reproduced the lock problem with your latest patch. I Hope that the following stacktrace will help to track what is going on. Unfortunately, I don't have the full thread stack traces dump, because they are logged in INFO level, which was not active at the time we reproduced the problem... But I do have the root cause, here it is: 2012-08-14 13:56:55,663 FelixDispatchQueue ERROR osgi.event.fwk - FrameworkEvent: ERROR java.lang.IllegalStateException: Could not obtain lock in 5000 milliseconds for component agent at org.apache.felix.scr.impl.manager.AbstractComponentManager.obtainReadLock(AbstractComponentManager.java:178) at org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:157) at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) at org.apache.felix.framework.Felix.registerService(Felix.java:3275) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:667) at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:653) at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:689) at org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1324) at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:562) at org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:293) at org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) at org.apache.felix.framework.Felix.registerService(Felix.java:3275) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) at com.alcatel.as.util.osgi.DependencyActivatorHelper.registerService(DependencyActivatorHelper.java:209) at com.alcatel.as.session.distributed.mgr.Engine$7.run(Engine.java:1370) at com.alcatel.as.service.concurrent.impl.Helper.runTask(Helper.java:54) at com.alcatel.as.service.concurrent.impl.QueueExecutor$1.run(QueueExecutor.java:165) at com.alcatel.as.service.concurrent.impl.SerialQueue.run(SerialQueue.java:80) at alcatel.tess.hometop.gateways.concurrent.ThreadPool$3.run(ThreadPool.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.Exception: Write lock stack trace for thread: Thread[ProxyAppReporter-FastCacheListener,5,main] at org.apache.felix.scr.impl.manager.AbstractComponentManager.escalateLock(AbstractComponentManager.java:205) at org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1332) at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:562) at org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:293) at org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) at org.apache.felix.framework.Felix.registerService(Felix.java:3275) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) at com.alcatel.as.service.config.impl.fc.PropertiesManager.update(PropertiesManager.java:211) at com.alcatel.as.service.config.impl.fc.FastCacheConfig.proxyAppPropertiesUpdated(FastCacheConfig.java:323) at com.nextenso.mgmt.reporter.ProxyAppReporter$FastCacheListeningThread.run(ProxyAppReporter.java:99) regards; /pierre On Sat, Aug 11, 2012 at 4:52 AM, David Jencks wrote: > I think this is a result of the changes in 3559. I added a test that I > think corresponds to your situation and made it work, see r1371859. I > think there might be some more possible simplifications, I'm looking into > it a bit more. Anyway let me know if there are still problems! > > I would have opened a new jira but it seems to be down, so I hope I'm > right about this being related to 3559 :-) > > thanks > david jencks > > On Aug 10, 2012, at 11:21 AM, David Jencks wrote: > > > Hi Pierre, > > > > I'm 99.9% sure this is a regression.... looking into it. > > > > IIUC you have a factory component set up where the instances the factory > creates are registered as services, and the stack trace shows another > component trying to get its reference to the newly created instance? > > > > I don't think there's an integration test for this scenario. > > > > thanks!! > > > > david jencks > > On Aug 10, 2012, at 5:45 AM, Pierre De Rop wrote: > > > >> Hi David; > >> > >> So I tested the latest scr commits but this time I then have another > >> different problem and my components don't startup: I'm now having the > >> following exception: > >> > >> (please see the last "Caused by: java.lang.IllegalStateException: > >> getServiceActive", at the end of the stacktraces) > >> > >> -> > >> > >> 2012-08-10 11:21:38,530 FelixDispatchQueue ERROR osgi.event.fwk - > >> FrameworkEvent: ERROR > >> > >> org.osgi.framework.ServiceException: Service factory exception: > >> getServiceActive > >> at > >> > org.apache.felix.framework.ServiceRegistrationImpl.getFactoryUnchecked(ServiceRegistrationImpl.java:312) > >> at > >> > org.apache.felix.framework.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:219) > >> at > >> > org.apache.felix.framework.ServiceRegistry.getService(ServiceRegistry.java:310) > >> at org.apache.felix.framework.Felix.getService(Felix.java:3420) > >> at > >> > org.apache.felix.framework.BundleContextImpl.getService(BundleContextImpl.java:468) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.getService(DependencyManager.java:822) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager$1.getInstance(DependencyManager.java:1097) > >> at > >> > org.apache.felix.scr.impl.helper.BindMethod.getParameters(BindMethod.java:562) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:225) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:603) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod$NotResolved.invoke(BaseMethod.java:560) > >> at > >> org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:484) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.invokeBindMethod(DependencyManager.java:1086) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.bind(DependencyManager.java:988) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.open(DependencyManager.java:924) > >> at > >> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createImplementationObject(ImmediateComponentManager.java:214) > >> at > >> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createComponent(ImmediateComponentManager.java:128) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1338) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:562) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:293) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >> at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >> at org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >> at > >> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:667) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:653) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:689) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1323) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:562) > >> at > >> > org.apache.felix.scr.impl.manager.ComponentFactoryImpl.newInstance(ComponentFactoryImpl.java:105) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.CompositeFactoryImpl.instantiateWithScope(CompositeFactoryImpl.java:246) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.CompositeFactoryImpl.access$300(CompositeFactoryImpl.java:33) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.CompositeFactoryImpl$1.run(CompositeFactoryImpl.java:204) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.SerialExecutor.execute(SerialExecutor.java:36) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.CompositeFactoryImpl.newInstance(CompositeFactoryImpl.java:192) > >> at > >> > com.alcatel_lucent.as.service.composite.impl.CompositeFactoryImpl.newInstance(CompositeFactoryImpl.java:186) > >> at com.nextenso.agent.Agent._bindCompositeFactory(Agent.java:959) > >> at com.nextenso.agent.Agent._activate(Agent.java:662) > >> at com.nextenso.agent.Agent.access$100(Agent.java:92) > >> at com.nextenso.agent.Agent$6.run(Agent.java:378) > >> at > >> > com.alcatel.as.service.concurrent.SerialExecutor.execute(SerialExecutor.java:41) > >> at com.nextenso.agent.Agent.activate(Agent.java:373) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > >> at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> at java.lang.reflect.Method.invoke(Method.java:597) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:226) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37) > >> at > >> > org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:603) > >> at > >> org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:484) > >> at > >> > org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:149) > >> at > >> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createImplementationObject(ImmediateComponentManager.java:240) > >> at > >> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createComponent(ImmediateComponentManager.java:128) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1338) > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:562) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:293) > >> at > >> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >> at > >> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >> at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >> at org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >> at > >> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >> at > >> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >> at > >> > com.alcatel.as.service.config.impl.fc.PropertiesManager.update(PropertiesManager.java:211) > >> at > >> > com.alcatel.as.service.config.impl.fc.FastCacheConfig.proxyAppPropertiesUpdated(FastCacheConfig.java:323) > >> at > >> > com.nextenso.mgmt.reporter.ProxyAppReporter$FastCacheListeningThread.run(ProxyAppReporter.java:99) > >> > >> Caused by: java.lang.IllegalStateException: getServiceActive > >> at > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager$State.getService(AbstractComponentManager.java:1114) > >> at > >> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.getService(ImmediateComponentManager.java:632) > >> at > >> > org.apache.felix.framework.ServiceRegistrationImpl.getFactoryUnchecked(ServiceRegistrationImpl.java:308) > >> ... 70 more > >> > >> > >> So, here an illegal state is thrown from AbstractComponentManager, line > >> 1114: > >> > >> Object getService( ImmediateComponentManager dcm ) > >> { > >> // log( dcm, "getService" ); > >> // return null; > >> throw new IllegalStateException("getService" + this); > >> } > >> > >> In order to get the actual state class name, I added a trace, just > before > >> throwing the illegal state, and I have seen that the currrent state is: > >> > org.apache.felix.scr.impl.manager.AbstractComponentManager$FactoryInstance. > >> > >> Is this a regression, or am I doing something wrong ? > >> > >> Thanks; > >> Pierre > >> > >> On Thu, Aug 9, 2012 at 6:21 PM, David Jencks > wrote: > >> > >>> OK, I went ahead and committed a bunch of work including a revised > locking > >>> scheme (which has been in a patch to FELIX-3456 for a couple months) > and > >>> the new attempt to track how locks are obtained and do a thread dump > when a > >>> lock problem occurs. I haven't been able to actually generate a lock > >>> problem to see if the logging code works.... > >>> > >>> The timeout should definitely be configurable, I'll try to get to that > >>> soon. > >>> > >>> Let me know if this causes problems!!! > >>> > >>> thanks > >>> david jencks > >>> > >>> On Aug 9, 2012, at 10:27 AM, Pierre De Rop wrote: > >>> > >>>> Hi David; > >>>> > >>>> It's also my case, I see the problem only occasionally, in some > automatic > >>>> tests, where many JVM might be starting concurrently, on the same > host. > >>>> > >>>> So, yes, If you commit the improved code, I could then retry my > automatic > >>>> tests, to see if we find something. > >>>> What is the improved code exactly ? > >>>> > >>>> Anyway, from my side, I have increased the state lock timeout to 60 > >>>> seconds, to see if the problem disappear. > >>>> ... don't you think that this timeout should be configurable, instead > of > >>>> using 5 seconds by default ? > >>>> > >>>> regards; > >>>> /Pierre > >>>> > >>>> On Thu, Aug 9, 2012 at 4:09 PM, David Jencks > >>> wrote: > >>>> > >>>>> I've been wondering if trunk code would show these deadlocks.... I > have > >>>>> some improved code in a jira and occasionally it also seems to show > this > >>>>> problem. (I've personally never seen it locally, it shows up > >>> occasionally > >>>>> in automated builds at work) I've added some more stuff that ought > to > >>> log > >>>>> appropriate info when this occurs, so since this has shown up "in the > >>> wild" > >>>>> I'm very tempted to commit the improved code and the (probably > >>> temporary) > >>>>> extra logging. > >>>>> > >>>>> Any objections? > >>>>> > >>>>> thanks > >>>>> david jencks > >>>>> > >>>>> On Aug 9, 2012, at 8:46 AM, Pierre De Rop wrote: > >>>>> > >>>>>> Hi everyone, > >>>>>> > >>>>>> I'm getting a "Could not obtain lock" exception from SCR. This > >>> exception > >>>>> is > >>>>>> thrown when SCR can't aquire its internal stack lock after 5 > seconds. > >>>>>> For now, I don't know if I am facing a bug from SCR, or if I'm just > in > >>> a > >>>>>> temporary situation where my JVM runs slowly (because of another > >>> process > >>>>>> which might consumes all CPUs temporarily). > >>>>>> > >>>>>> However, I provide here the exception, as well as a full dump stack > >>>>> trace, > >>>>>> in case you find out a possible deadlock from SCR ? > >>>>>> (I'm using SCR from trunk, and Framework 4.0.3): > >>>>>> > >>>>>> java.lang.IllegalStateException: Could not obtain lock > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.obtainStateLock(AbstractComponentManager.java:161) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:158) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> at > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> at org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:603) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:626) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1253) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:291) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> at > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> at org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> at com.nextenso.agent.Agent.initSuperAgent(Agent.java:715) > >>>>>> at com.nextenso.agent.Agent._activate(Agent.java:646) > >>>>>> at com.nextenso.agent.Agent.access$100(Agent.java:92) > >>>>>> at com.nextenso.agent.Agent$5.run(Agent.java:365) > >>>>>> at > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.SerialExecutor.execute(SerialExecutor.java:41) > >>>>>> at com.nextenso.agent.Agent.bindSessionManager(Agent.java:362) > >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>>>> at > >>>>>> > >>>>> > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >>>>>> at > >>>>>> > >>>>> > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>> at java.lang.reflect.Method.invoke(Method.java:616) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:226) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:603) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod$NotResolved.invoke(BaseMethod.java:560) > >>>>>> at > >>>>>> > org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:484) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.invokeBindMethod(DependencyManager.java:1084) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:326) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> at > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> at org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> at > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> at > >>>>>> > >>>>> > >>> > com.alcatel.as.util.osgi.DependencyActivatorHelper.registerService(DependencyActivatorHelper.java:205) > >>>>>> at > >>>>> com.alcatel.as.session.distributed.mgr.Engine$7.run(Engine.java:1401) > >>>>>> at > >>>>> com.alcatel.as.service.concurrent.impl.Helper.runTask(Helper.java:54) > >>>>>> at > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.impl.QueueExecutor$1.run(QueueExecutor.java:165) > >>>>>> at > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.impl.SerialQueue.run(SerialQueue.java:80) > >>>>>> at > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.concurrent.ThreadPool$3.run(ThreadPool.java:286) > >>>>>> at > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >>>>>> at > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> at java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> So, to investigate, I have modified the > >>>>>> AbstractComponentManager.obtainStateLock, in order to generate a > dump > >>> of > >>>>>> all threads, when the problem occurs: > >>>>>> > >>>>>> public final void obtainStateLock() > >>>>>> { > >>>>>> try > >>>>>> { > >>>>>> if (!m_stateLock.tryLock( m_timeout ) ) > >>>>>> { > >>>>>> dumpThreads(); > >>>>>> throw new IllegalStateException( "Could not obtain lock" > >>>>> ); > >>>>>> } > >>>>>> } > >>>>>> catch ( InterruptedException e ) > >>>>>> { > >>>>>> //TODO this is so wrong > >>>>>> throw new IllegalStateException( "Could not obtain lock > >>>>>> (Reason: " + e + ")" ); > >>>>>> } > >>>>>> } > >>>>>> > >>>>>> private void dumpThreads() { > >>>>>> StringWriter sw = new StringWriter(); > >>>>>> sw.write("SCR: : Could not obtain lock timely: dump all > >>>>> threads\n"); > >>>>>> try { > >>>>>> Map mapStacks = Thread.getAllStackTraces(); > >>>>>> Iterator threads = mapStacks.keySet().iterator(); > >>>>>> while (threads.hasNext()) { > >>>>>> Thread thread = (Thread) threads.next(); > >>>>>> StackTraceElement[] stes = (StackTraceElement[]) > >>>>>> mapStacks.get(thread); > >>>>>> sw.write("\nThread [" + thread.getName() + " prio=" + > >>>>>> thread.getPriority() > >>>>>> + ", state=" + thread.getState() + "] --> StackTrace > >>> elements > >>>>>> ...\n"); > >>>>>> for (int i = 0; i < stes.length; i ++) { > >>>>>> StackTraceElement ste = stes[i]; > >>>>>> sw.write("\t" + ste.toString() + "\n"); > >>>>>> } > >>>>>> } > >>>>>> > >>>>>> Activator.log(LogService.LOG_ERROR, null, sw.toString(), > null); > >>>>>> } catch (Throwable t) { > >>>>>> Activator.log(LogService.LOG_ERROR, null, "Could not obtain > >>> lock > >>>>>> timely, and could not dump threads", t); > >>>>>> } > >>>>>> } > >>>>>> > >>>>>> And when the problem occurs, I then have the following thread dump > >>> stacks > >>>>>> ... As far as I can see in the stack traces, and also from the SCR > >>> code, > >>>>>> the method AbstractComponentManager.obtainStateLock method is always > >>>>>> invoked using a try/finally clause, where the lock is realeased in > the > >>>>>> finally block ... So, I wonder how it is possible to be blocked in > the > >>>>>> obtainStateLock ... unless the JVM is temporarily halted for more > than > >>> 5 > >>>>>> seconds, because another process takes too much CPUS ... > >>>>>> > >>>>>> In the latter case, may be it could help if the stackLock timeout > could > >>>>> be > >>>>>> configured (because for now, it is hard coded to 5000 millis ... > what > >>> do > >>>>>> you think ?) > >>>>>> > >>>>>> Here are all threads dump: > >>>>>> > >>>>>> Thread [IO-ThreadPool-3 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-8 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [FelixStartLevel prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:272) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [ReporterListener-18 prio=5, state=WAITING] --> StackTrace > >>>>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.dbase.api.FastCacheCompatibility$Listener.getNextEntry(FastCacheCompatibility.java:267) > >>>>>> com.nextenso.dbase.api.FastCache.getEvent(FastCache.java:948) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ListeningThread.run(FastCacheReporter.java:665) > >>>>>> > >>>>>> Thread [CM Event Dispatcher prio=5, state=WAITING] --> StackTrace > >>>>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:77) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [FastCache-selector prio=5, state=RUNNABLE] --> StackTrace > >>>>> elements > >>>>>> ... > >>>>>> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > >>>>>> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228) > >>>>>> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) > >>>>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.reactor.impl.ReactorImpl.run(ReactorImpl.java:294) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-4 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [SessionRegistry-selector prio=5, state=RUNNABLE] --> > StackTrace > >>>>>> elements ... > >>>>>> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > >>>>>> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228) > >>>>>> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) > >>>>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.reactor.impl.ReactorImpl.run(ReactorImpl.java:294) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-0 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [CM Configuration Updater prio=5, state=WAITING] --> > StackTrace > >>>>>> elements ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:77) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [SCR Component Actor prio=5, state=WAITING] --> StackTrace > >>>>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.ComponentActorThread.run(ComponentActorThread.java:76) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Finalizer prio=8, state=WAITING] --> StackTrace elements ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) > >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) > >>>>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177) > >>>>>> > >>>>>> Thread [Processing-ThreadPool-10 prio=5, state=WAITING] --> > StackTrace > >>>>>> elements ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [FastCache-reactor prio=5, state=WAITING] --> StackTrace > >>> elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Reporting prio=5, state=TIMED_WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> com.nextenso.mgmt.reporter.Pause.pause(Pause.java:24) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ReportingThread.pause(FastCacheReporter.java:475) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ReportingThread.run(FastCacheReporter.java:534) > >>>>>> > >>>>>> Thread [WheelTimerService prio=5, state=TIMED_WAITING] --> > StackTrace > >>>>>> elements ... > >>>>>> java.lang.Thread.sleep(Native Method) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.impl.WheelTimerServiceImpl.run(WheelTimerServiceImpl.java:137) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-9 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Processing-ThreadPool-11 prio=5, state=RUNNABLE] --> > StackTrace > >>>>>> elements ... > >>>>>> java.lang.Thread.dumpThreads(Native Method) > >>>>>> java.lang.Thread.getAllStackTraces(Thread.java:1546) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.dumpThreads(AbstractComponentManager.java:175) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.obtainStateLock(AbstractComponentManager.java:160) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:158) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:603) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:626) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1253) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:291) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> com.nextenso.agent.Agent.initSuperAgent(Agent.java:715) > >>>>>> com.nextenso.agent.Agent._activate(Agent.java:646) > >>>>>> com.nextenso.agent.Agent.access$100(Agent.java:92) > >>>>>> com.nextenso.agent.Agent$5.run(Agent.java:365) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.SerialExecutor.execute(SerialExecutor.java:41) > >>>>>> com.nextenso.agent.Agent.bindSessionManager(Agent.java:362) > >>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>> java.lang.reflect.Method.invoke(Method.java:616) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:226) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:603) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod$NotResolved.invoke(BaseMethod.java:560) > >>>>>> > >>>>> > org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:484) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.invokeBindMethod(DependencyManager.java:1084) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:326) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.util.osgi.DependencyActivatorHelper.registerService(DependencyActivatorHelper.java:205) > >>>>>> > com.alcatel.as.session.distributed.mgr.Engine$7.run(Engine.java:1401) > >>>>>> > com.alcatel.as.service.concurrent.impl.Helper.runTask(Helper.java:54) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.impl.QueueExecutor$1.run(QueueExecutor.java:165) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.impl.SerialQueue.run(SerialQueue.java:80) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.concurrent.ThreadPool$3.run(ThreadPool.java:286) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-2 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Session-reactor prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [ControlFlow prio=5, state=WAITING] --> StackTrace elements > ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> java.util.TimerThread.mainLoop(Timer.java:505) > >>>>>> java.util.TimerThread.run(Timer.java:484) > >>>>>> > >>>>>> Thread [FelixDispatchQueue prio=5, state=WAITING] --> StackTrace > >>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:1063) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:101) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Signal Dispatcher prio=9, state=RUNNABLE] --> StackTrace > >>> elements > >>>>>> ... > >>>>>> > >>>>>> Thread [ReporterListener-12 prio=5, state=WAITING] --> StackTrace > >>>>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.dbase.api.FastCacheCompatibility$Listener.getNextEntry(FastCacheCompatibility.java:267) > >>>>>> com.nextenso.dbase.api.FastCache.getEvent(FastCache.java:948) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ListeningThread.run(FastCacheReporter.java:665) > >>>>>> > >>>>>> Thread [IO-ThreadPool-7 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [JdkTimerService-1 prio=5, state=TIMED_WAITING] --> > StackTrace > >>>>>> elements ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> > >>> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081) > >>>>>> java.util.concurrent.DelayQueue.take(DelayQueue.java:193) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Stdout prio=5, state=WAITING] --> StackTrace elements ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.log4j.impl.Log4jConfigurator$StdoutRedirectorThread.run(Log4jConfigurator.java:356) > >>>>>> > >>>>>> Thread [IO-ThreadPool-5 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [SessionRegistry-reactor prio=5, state=TIMED_WAITING] --> > >>>>> StackTrace > >>>>>> elements ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> > >>>>> > com.alcatel.as.distributed.lock.FCDBImpl$Lock.doWait(FCDBImpl.java:258) > >>>>>> > com.alcatel.as.distributed.lock.FCDBImpl$Lock.lock(FCDBImpl.java:234) > >>>>>> com.alcatel.as.distributed.lock.FCDBImpl.lock(FCDBImpl.java:157) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.session.distributed.registry.AbstractRegistryMonitor.getLock(AbstractRegistryMonitor.java:395) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.session.distributed.registry.InternalRegistryMonitor.getLock(InternalRegistryMonitor.java:49) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.session.distributed.registry.AbstractRegistryMonitor._start(AbstractRegistryMonitor.java:187) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.session.distributed.registry.AbstractRegistryMonitor$2.run(AbstractRegistryMonitor.java:253) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.reactor.impl.ReactorImpl$2.run(ReactorImpl.java:426) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.reactor.impl.ReactorExecutor.run(ReactorExecutor.java:90) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-1 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [IO-ThreadPool-6 prio=5, state=WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Thread-22 prio=5, state=TIMED_WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> EDU.oswego.cs.dl.util.concurrent.LinkedQueue.poll(Unknown Source) > >>>>>> EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(Unknown > >>>>> Source) > >>>>>> EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown > >>>>>> Source) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Reporting prio=5, state=TIMED_WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> com.nextenso.mgmt.reporter.Pause.pause(Pause.java:24) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ReportingThread.pause(FastCacheReporter.java:475) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ReportingThread.run(FastCacheReporter.java:534) > >>>>>> > >>>>>> Thread [Timer-0 prio=5, state=TIMED_WAITING] --> StackTrace elements > >>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.util.TimerThread.mainLoop(Timer.java:531) > >>>>>> java.util.TimerThread.run(Timer.java:484) > >>>>>> > >>>>>> Thread [ProxyAppReporter-FastCacheListener prio=5, > state=TIMED_WAITING] > >>>>> --> > >>>>>> StackTrace elements ... > >>>>>> sun.misc.Unsafe.park(Native Method) > >>>>>> > >>> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:933) > >>>>>> > >>>>>> > >>>>> > >>> > java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247) > >>>>>> > >>>>> > java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:444) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$JLock.tryLock(AbstractComponentManager.java:1537) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.obtainStateLock(AbstractComponentManager.java:158) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:158) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:603) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:626) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1253) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:291) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.util.osgi.DependencyActivatorHelper.registerService(DependencyActivatorHelper.java:205) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.proxylet.deployer.impl.PxletDeployerImpl.registerWhenReady(PxletDeployerImpl.java:378) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.proxylet.deployer.impl.PxletDeployerImpl.access$400(PxletDeployerImpl.java:41) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.proxylet.deployer.impl.PxletDeployerImpl$7.run(PxletDeployerImpl.java:172) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.concurrent.SerialExecutor.execute(SerialExecutor.java:41) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.proxylet.deployer.impl.PxletDeployerImpl.bindObject(PxletDeployerImpl.java:149) > >>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>> java.lang.reflect.Method.invoke(Method.java:616) > >>>>>> > >>>>> > org.apache.felix.dm.InvocationUtil.invokeMethod(InvocationUtil.java:80) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.InvocationUtil.invokeCallbackMethod(InvocationUtil.java:50) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.impl.ComponentImpl.invokeCallbackMethod(ComponentImpl.java:668) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.impl.dependencies.ServiceDependencyImpl.invoke(ServiceDependencyImpl.java:555) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.impl.dependencies.ServiceDependencyImpl.invokeAdded(ServiceDependencyImpl.java:534) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.impl.dependencies.ServiceDependencyImpl.addedService(ServiceDependencyImpl.java:465) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.tracker.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1317) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.tracker.AbstractTracked.trackAdding(AbstractTracked.java:290) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.tracker.AbstractTracked.track(AbstractTracked.java:236) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.tracker.ServiceTracker$Tracked.serviceChangedHideAspects(ServiceTracker.java:1198) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.dm.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1093) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:603) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:626) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1253) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:291) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:603) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:626) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1253) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.ComponentFactoryImpl.newInstance(ComponentFactoryImpl.java:105) > >>>>>> > >>>>>> > >>> > com.alcatel.xcap.client.impl.SPITracker.configureSPI(SPITracker.java:231) > >>>>>> com.alcatel.xcap.client.impl.SPITracker.start(SPITracker.java:149) > >>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >>>>>> > >>>>>> > >>>>> > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>> java.lang.reflect.Method.invoke(Method.java:616) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:226) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:603) > >>>>>> > >>>>> > org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:484) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:149) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createImplementationObject(ImmediateComponentManager.java:225) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.ImmediateComponentManager.createComponent(ImmediateComponentManager.java:121) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1259) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:508) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:291) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:171) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) > >>>>>> org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4260) > >>>>>> org.apache.felix.framework.Felix.registerService(Felix.java:3275) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) > >>>>>> > >>>>>> > >>>>> > >>> > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.config.impl.fc.PropertiesManager.update(PropertiesManager.java:211) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.as.service.config.impl.fc.FastCacheConfig.proxyAppPropertiesUpdated(FastCacheConfig.java:323) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.ProxyAppReporter$FastCacheListeningThread.run(ProxyAppReporter.java:99) > >>>>>> > >>>>>> Thread [Session-selector prio=5, state=RUNNABLE] --> StackTrace > >>> elements > >>>>> ... > >>>>>> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > >>>>>> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228) > >>>>>> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) > >>>>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) > >>>>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) > >>>>>> > >>>>>> > >>>>> > >>> > alcatel.tess.hometop.gateways.reactor.impl.ReactorImpl.run(ReactorImpl.java:294) > >>>>>> java.lang.Thread.run(Thread.java:679) > >>>>>> > >>>>>> Thread [Reference Handler prio=10, state=WAITING] --> StackTrace > >>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) > >>>>>> > >>>>>> Thread [main prio=5, state=WAITING] --> StackTrace elements ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79) > >>>>>> org.apache.felix.framework.Felix.waitForStop(Felix.java:921) > >>>>>> org.apache.felix.main.Main.main(Main.java:299) > >>>>>> com.alcatel.as.felix.FelixLauncher.main(FelixLauncher.java:69) > >>>>>> > >>>>>> Thread [ReporterListener-7 prio=5, state=WAITING] --> StackTrace > >>> elements > >>>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.dbase.api.FastCacheCompatibility$Listener.getNextEntry(FastCacheCompatibility.java:267) > >>>>>> com.nextenso.dbase.api.FastCache.getEvent(FastCache.java:948) > >>>>>> > >>>>>> > >>>>> > >>> > com.nextenso.mgmt.reporter.FastCacheReporter$ListeningThread.run(FastCacheReporter.java:665) > >>>>>> > >>>>>> Thread [MySQL Statement Cancellation Timer prio=5, state=WAITING] > --> > >>>>>> StackTrace elements ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> java.lang.Object.wait(Object.java:502) > >>>>>> java.util.TimerThread.mainLoop(Timer.java:505) > >>>>>> java.util.TimerThread.run(Timer.java:484) > >>>>>> > >>>>>> Thread [Thread-29 prio=5, state=TIMED_WAITING] --> StackTrace > elements > >>>>> ... > >>>>>> java.lang.Object.wait(Native Method) > >>>>>> > >>>>>> > >>>>> > >>> > com.alcatel.presence.useridmgr.jdbc.UserIdJdbcPoolConnector$UserIdJdbcPoolConnectorCleaner.run(UserIdJdbcPoolConnector.java:563) > >>>>>> > >>>>>> > >>>>>> thanks for your help; > >>>>>> > >>>>>> kind regards; > >>>>>> /Pierre > >>>>> > >>>>> > >>> > >>> > > > > --485b397dd729bdbd0f04c8038328--