portals-jetspeed-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sean Taylor <da...@bluesunrise.com>
Subject Re: [jira] Commented: (JS2-682) Jetspeed thread waiting to lock for infinity time
Date Thu, 19 Apr 2007 01:19:39 GMT
I knew about this ... found it in performance testing last week
Will commit a patch within the next 24 hours

On Apr 17, 2007, at 2:11 PM, Vitaly Baranovsky (JIRA) wrote:

>
>     [ https://issues.apache.org/jira/browse/JS2-682? 
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment- 
> tabpanel#action_12489378 ]
>
> Vitaly Baranovsky commented on JS2-682:
> ---------------------------------------
>
> So, what can Jetspeed people say about this waiting to lock  for  
> infinity time?
>
>> Jetspeed thread waiting to lock for infinity time
>> -------------------------------------------------
>>
>>                 Key: JS2-682
>>                 URL: https://issues.apache.org/jira/browse/JS2-682
>>             Project: Jetspeed 2
>>          Issue Type: Bug
>>          Components: Components Core
>>    Affects Versions: 2.1-dev
>>         Environment: Apache Tomcat 5.5.20; Suse Linux 10.1; log4j  
>> 1.2.11
>>            Reporter: Vitaly Baranovsky
>>         Attachments: stackdump.zip
>>
>>
>> Sometimes our production server falls down.
>> I have analyzed full thread stack dump after fall down, and I have  
>> found that server falls down because of one of thread waits for  
>> synchronized code in IsolatedLog4JLogger.getLogger  
>> (<0x00002b35dd5ba6c8>). And other threads are waiting for lock  
>> that was locked by this thread (<0x00002b35ec0aa1e8>).
>> The dump of this thread (in infinitly waiting state):
>> "http-8080-Processor499" daemon prio=1 tid=0x00002aaab0a38d10  
>> nid=0x23b3 waiting for monitor entry  
>> [0x00000000554a0000..0x00000000554a2da0]
>>     at  
>> org.apache.jetspeed.webapp.logging.IsolatedLog4JLogger.getLogger 
>> (IsolatedLog4JLogger.java:158)
>>     - waiting to lock <0x00002b35dd5ba6c8> (a java.lang.Class)
>>     at org.apache.jetspeed.webapp.logging.IsolatedLog4JLogger.debug 
>> (IsolatedLog4JLogger.java:166)
>>     at  
>> org.springframework.transaction.support.AbstractPlatformTransactionMa 
>> nager.triggerAfterCompletion 
>> (AbstractPlatformTransactionManager.java:568)
>>     at  
>> org.springframework.transaction.support.AbstractPlatformTransactionMa 
>> nager.commit(AbstractPlatformTransactionManager.java:429)
>>     at  
>> org.springframework.transaction.interceptor.TransactionAspectSupport. 
>> doCommitTransactionAfterReturning(TransactionAspectSupport.java:243)
>>     at  
>> org.springframework.transaction.interceptor.TransactionInterceptor.in 
>> voke(TransactionInterceptor.java:66)
>>     at  
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed( 
>> ReflectiveMethodInvocation.java:144)
>>     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke 
>> (JdkDynamicAopProxy.java:174)
>>     at $Proxy7.getChildren(Unknown Source)
>>     at  
>> org.apache.jetspeed.prefs.impl.PreferencesImpl.childrenNamesSpi 
>> (PreferencesImpl.java:129)
>>     at java.util.prefs.AbstractPreferences.childrenNames 
>> (AbstractPreferences.java:699)
>>     - locked <0x00002b35ec0aa1e8> (a java.lang.Object)
>>     at  
>> org.apache.jetspeed.om.preference.impl.PrefsPreferenceSetImpl.size 
>> (PrefsPreferenceSetImpl.java:271)
>>     at  
>> org.apache.jetspeed.om.preference.impl.PrefsPreferenceSetImpl 
>> $PortletPrefsIterator.hasNext(PrefsPreferenceSetImpl.java:364)
>>     at  
>> org.apache.jetspeed.om.preference.impl.FragmentPortletPreferenceSet.< 
>> init>(FragmentPortletPreferenceSet.java:62)
>>     at  
>> org.apache.jetspeed.om.portlet.impl.FragmentPortletDefinition.getPref 
>> erenceSet(FragmentPortletDefinition.java:229)
>>     at org.apache.pluto.core.impl.PortletPreferencesImpl.<init> 
>> (PortletPreferencesImpl.java:73)
>>     at  
>> org.apache.pluto.factory.impl.PortletPreferencesFactoryImpl.getPortle 
>> tPreferences(PortletPreferencesFactoryImpl.java:39)
>>     at  
>> org.apache.pluto.factory.PortletObjectAccess.getPortletPreferences 
>> (PortletObjectAccess.java:127)
>>     at org.apache.pluto.core.impl.RenderRequestImpl.getPreferences 
>> (RenderRequestImpl.java:74)
>>     at $PortletRequest_111c59420ea.getPreferences 
>> ($PortletRequest_111c59420ea.java)
>>     at $PortletRequest_111c59420e7.getPreferences 
>> ($PortletRequest_111c59420e7.java)
>>     at  
>> net.mycompany.portal.news.newslist.NewsListPage.pageBeginRender 
>> (NewsListPage.java:792)
>>     at org.apache.tapestry.AbstractPage.firePageBeginRender 
>> (AbstractPage.java:478)
>>     at org.apache.tapestry.AbstractPage.renderPage 
>> (AbstractPage.java:268)
>>     at org.apache.tapestry.engine.RequestCycle.renderPage 
>> (RequestCycle.java:366)
>>     at org.apache.tapestry.portlet.PortletRendererImpl.renderPage 
>> (PortletRendererImpl.java:76)
>>     at $PortletRenderer_111c59420d9.renderPage 
>> ($PortletRenderer_111c59420d9.java)
>>     at org.apache.tapestry.portlet.PortletHomeService.service 
>> (PortletHomeService.java:80)
>>     at $IEngineService_111c59420b9.service 
>> ($IEngineService_111c59420b9.java)
>>     at  
>> org.apache.tapestry.services.impl.EngineServiceOuterProxy.service 
>> (EngineServiceOuterProxy.java:66)
>>     at org.apache.tapestry.engine.AbstractEngine.service 
>> (AbstractEngine.java:248)
>>     at  
>> org.apache.tapestry.services.impl.InvokeEngineTerminator.service 
>> (InvokeEngineTerminator.java:60)
>>     at $WebRequestServicer_111c5942059.service 
>> ($WebRequestServicer_111c5942059.java)
>>     at $WebRequestServicer_111c5942057.service 
>> ($WebRequestServicer_111c5942057.java)
>>     at  
>> org.apache.tapestry.portlet.RenderRequestServicerToWebRequestServicer 
>> Bridge.service 
>> (RenderRequestServicerToWebRequestServicerBridge.java:49)
>>     at $RenderRequestServicer_111c5942051.service 
>> ($RenderRequestServicer_111c5942051.java)
>>     at $RenderRequestServicer_111c594204b.service 
>> ($RenderRequestServicer_111c594204b.java)
>>     at org.apache.tapestry.portlet.ApplicationPortlet.render 
>> (ApplicationPortlet.java:161)
>>     at org.apache.jetspeed.factory.JetspeedPortletInstance.render 
>> (JetspeedPortletInstance.java:102)
>>     at org.apache.jetspeed.container.JetspeedContainerServlet.doGet 
>> (JetspeedContainerServlet.java:230)
>>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>     at  
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
>> (ApplicationFilterChain.java:252)
>>     at org.apache.catalina.core.ApplicationFilterChain.doFilter 
>> (ApplicationFilterChain.java:173)
>>     at org.apache.catalina.core.ApplicationDispatcher.invoke 
>> (ApplicationDispatcher.java:672)
>>     at org.apache.catalina.core.ApplicationDispatcher.doInclude 
>> (ApplicationDispatcher.java:574)
>>     at org.apache.catalina.core.ApplicationDispatcher.include 
>> (ApplicationDispatcher.java:499)
>>     at  
>> org.apache.jetspeed.container.invoker.ServletPortletInvoker.invoke 
>> (ServletPortletInvoker.java:215)
>>     at  
>> org.apache.jetspeed.container.invoker.ServletPortletInvoker.render 
>> (ServletPortletInvoker.java:126)
>>     at org.apache.pluto.PortletContainerImpl.renderPortlet 
>> (PortletContainerImpl.java:119)
>>     at  
>> org.apache.jetspeed.container.JetspeedPortletContainerWrapper.renderP 
>> ortlet(JetspeedPortletContainerWrapper.java:120)
>>     at org.apache.jetspeed.aggregator.impl.RenderingJobImpl.execute 
>> (RenderingJobImpl.java:121)
>>     at  
>> org.apache.jetspeed.aggregator.impl.PortletRendererImpl.renderNow 
>> (PortletRendererImpl.java:120)
>>     at  
>> org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.aggregateAndRe 
>> nder(PageAggregatorImpl.java:199)
>>     at  
>> org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.aggregateAndRe 
>> nder(PageAggregatorImpl.java:182)
>>     at  
>> org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.aggregateAndRe 
>> nder(PageAggregatorImpl.java:182)
>>     at org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.build 
>> (PageAggregatorImpl.java:106)
>>     at org.apache.jetspeed.aggregator.AggregatorValve.invoke 
>> (AggregatorValve.java:48)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at org.apache.jetspeed.decoration.DecorationValve.invoke 
>> (DecorationValve.java:110)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.pipeline.valve.impl.ActionValveImpl.invoke 
>> (ActionValveImpl.java:147)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at org.apache.jetspeed.container.ContainerValve.invoke 
>> (ContainerValve.java:76)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at org.apache.jetspeed.profiler.impl.ProfilerValveImpl.invoke 
>> (ProfilerValveImpl.java:255)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.security.impl.LoginValidationValveImpl.invoke 
>> (LoginValidationValveImpl.java:159)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.security.impl.PasswordCredentialValveImpl.invoke( 
>> PasswordCredentialValveImpl.java:149)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.localization.impl.LocalizationValveImpl.invoke 
>> (LocalizationValveImpl.java:169)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at org.apache.jetspeed.security.impl.AbstractSecurityValve 
>> $1.run(AbstractSecurityValve.java:117)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at javax.security.auth.Subject.doAsPrivileged(Subject.java:454)
>>     at  
>> org.apache.jetspeed.security.impl.AbstractSecurityValve.invoke 
>> (AbstractSecurityValve.java:111)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.container.url.impl.PortalURLValveImpl.invoke 
>> (PortalURLValveImpl.java:67)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at  
>> org.apache.jetspeed.capabilities.impl.CapabilityValveImpl.invoke 
>> (CapabilityValveImpl.java:128)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline 
>> $Invocation.invokeNext(JetspeedPipeline.java:166)
>>     at org.apache.jetspeed.pipeline.JetspeedPipeline.invoke 
>> (JetspeedPipeline.java:145)
>>     at org.apache.jetspeed.engine.JetspeedEngine.service 
>> (JetspeedEngine.java:214)
>>     at org.apache.jetspeed.engine.JetspeedServlet.doGet 
>> (JetspeedServlet.java:238)
>>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>     at  
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
>> (ApplicationFilterChain.java:252)
>>     at org.apache.catalina.core.ApplicationFilterChain.doFilter 
>> (ApplicationFilterChain.java:173)
>>     at com.opensymphony.oscache.web.filter.CacheFilter.doFilter 
>> (CacheFilter.java:168)
>>     at  
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
>> (ApplicationFilterChain.java:202)
>>     at org.apache.catalina.core.ApplicationFilterChain.doFilter 
>> (ApplicationFilterChain.java:173)
>>     at org.apache.catalina.core.StandardWrapperValve.invoke 
>> (StandardWrapperValve.java:213)
>>     at org.apache.catalina.core.StandardContextValve.invoke 
>> (StandardContextValve.java:178)
>>     at org.apache.catalina.authenticator.AuthenticatorBase.invoke 
>> (AuthenticatorBase.java:432)
>>     at org.apache.catalina.core.StandardHostValve.invoke 
>> (StandardHostValve.java:126)
>>     at org.apache.catalina.valves.ErrorReportValve.invoke 
>> (ErrorReportValve.java:105)
>>     at org.apache.catalina.core.StandardEngineValve.invoke 
>> (StandardEngineValve.java:107)
>>     at org.apache.catalina.connector.CoyoteAdapter.service 
>> (CoyoteAdapter.java:148)
>>     at org.apache.coyote.http11.Http11Processor.process 
>> (Http11Processor.java:869)
>>     at org.apache.coyote.http11.Http11BaseProtocol 
>> $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 
>> 664)
>>     at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket 
>> (PoolTcpEndpoint.java:527)
>>     at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt 
>> (LeaderFollowerWorkerThread.java:80)
>>     at org.apache.tomcat.util.threads.ThreadPool 
>> $ControlRunnable.run(ThreadPool.java:684)
>>     at java.lang.Thread.run(Thread.java:595)
>> So, this thread is waiting for <0x00002b35dd5ba6c8> lock, but  
>> there is no one other thread use this lock! (Full thread stack  
>> dump is attached).
>> Method IsolatedLog4JLogger.getLogger, when thread inifinitly  
>> waiting for synchronized object::
>>    private Log4JLogger getLogger()
>>     {
>>         synchronized (IsolatedLog4JLogger.class)
>>         {
>>             return logger;
>>         }
>>     }
>> IsolatedLog4JLogger class with all parts of code, that use  
>> "synchronized (IsolatedLog4JLogger.class)":
>> public class IsolatedLog4JLogger implements Log
>> {
>>     private static Hierarchy hierarchy;
>>     private static HashMap notIsolatedLoggers = new HashMap();
>>
>>     private Log4JLogger logger; // the wrapped Log4JLogger
>>     public static void setHierarchy(Hierarchy hierarchy)
>>     {
>>         synchronized (IsolatedLog4JLogger.class)
>>         {
>>             if ( IsolatedLog4JLogger.hierarchy == null )
>>             {
>>                 IsolatedLog4JLogger.hierarchy = hierarchy;
>>                 if ( notIsolatedLoggers.size() > 0 )
>>                 {
>>                     // Reroute existing IsolatedLog4JLogger instances
>>                     // which were created before the new  
>> LoggerRepository.
>>                     // Note: This situation should be prevented as  
>> much as
>>                     //       possible by calling setHierarchy from
>>                     //       a ServletContextListener or a Servlet  
>> its init method
>>                     //       which has a load-on-startup value of  
>> 0 (zero).
>>                     Iterator iter = notIsolatedLoggers.entrySet 
>> ().iterator();
>>                     while (iter.hasNext())
>>                     {
>>                         Map.Entry entry = (Map.Entry)iter.next();
>>                         IsolatedLog4JLogger logger =  
>> (IsolatedLog4JLogger)entry.getKey();
>>                         logger.setLogger(new Log4JLogger 
>> (hierarchy.getLogger((String)entry.getValue())));
>>                     }
>>                 }
>>                 notIsolatedLoggers = null;
>>             }
>>         }
>>     }
>>
>>     public IsolatedLog4JLogger(String name)
>>     {
>>         synchronized (IsolatedLog4JLogger.class)
>>         {
>>             if ( hierarchy == null )
>>             {
>>                 // A LogFactory.getLog(name) is called before
>>                 // our ContextClassLoader Hierarchy could be set.
>>                 // Temporarily save this instance so it can be
>>                 // rerouted one the Hierarchy is set.
>>                 logger = new Log4JLogger(name);
>>                 notIsolatedLoggers.put(this,name);
>>             }
>>             else
>>             {
>>                 logger = new Log4JLogger(hierarchy.getLogger(name));
>>             }
>>         }
>>     }
>>     ................
>> }
>
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jetspeed-dev-unsubscribe@portals.apache.org
> For additional commands, e-mail: jetspeed-dev-help@portals.apache.org
>
>

-- 
David Sean Taylor
Bluesunrise Software
david@bluesunrise.com
[office] +01 707 773-4646
[mobile] +01 707 529 9194



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message