Return-Path: X-Original-To: apmail-myfaces-users-archive@www.apache.org Delivered-To: apmail-myfaces-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B42E44267 for ; Mon, 11 Jul 2011 20:21:49 +0000 (UTC) Received: (qmail 60513 invoked by uid 500); 11 Jul 2011 20:21:48 -0000 Delivered-To: apmail-myfaces-users-archive@myfaces.apache.org Received: (qmail 60470 invoked by uid 500); 11 Jul 2011 20:21:48 -0000 Mailing-List: contact users-help@myfaces.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "MyFaces Discussion" Delivered-To: mailing list users@myfaces.apache.org Received: (qmail 60462 invoked by uid 99); 11 Jul 2011 20:21:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2011 20:21:48 +0000 X-ASF-Spam-Status: No, hits=-0.6 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lu4242@gmail.com designates 209.85.218.53 as permitted sender) Received: from [209.85.218.53] (HELO mail-yi0-f53.google.com) (209.85.218.53) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2011 20:21:43 +0000 Received: by yic15 with SMTP id 15so1956608yic.12 for ; Mon, 11 Jul 2011 13:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=1gyaVTEOKPBApn4ojl2xODgivP9dHXpcLCXhFtmezko=; b=XqgPB0WLh0SxKK1Oeb6b56WR9KjwIjlKEigQlvTpQyiRwRIQQdchV7giH0B9j8V2rZ 11SUbtj5TeBZ987TSQ27EjSACqOXfZwaUSM+yHacr1aBhsCfeR1FYbr3FEVwTOpiQKb7 qZ1x+GzsvHfbHwkfkyqLhvh9tuZ3wjAbw13Z8= MIME-Version: 1.0 Received: by 10.142.63.7 with SMTP id l7mr2258525wfa.282.1310415681948; Mon, 11 Jul 2011 13:21:21 -0700 (PDT) Received: by 10.143.159.6 with HTTP; Mon, 11 Jul 2011 13:21:21 -0700 (PDT) In-Reply-To: References: <1309807748.3180.131.camel@markoc-desktop> <1309848334.80617.YahooMailClassic@web27807.mail.ukl.yahoo.com> Date: Mon, 11 Jul 2011 15:21:21 -0500 Message-ID: Subject: Re: MyFaces - Thread issues under High load From: Leonardo Uribe To: MyFaces Discussion Cc: rajadurai_p@infosys.com Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable Hi I have attached some patches on MYFACES-3216 and commited the code on trunk and related branches. Please let us know if the code proposed fix the problem you mention. Leonardo 2011/7/11 Leonardo Uribe : > Hi > > I have created: > > https://issues.apache.org/jira/browse/MYFACES-3216 > > to deal with this code. I checked it and there is a potential risk of > exceptions when map.put is called. Maybe that is the source of your > problem, but it could be better if you can give us more info about it. > > Leonardo Uribe > > 2011/7/5 Mark Struberg : >> Most importantly: which JDK/JRE are you running WebLogic under? And whic= h weblogic version do you use? >> >> You have definitely checked diskfree? (df -h) and mem is also no concern= ? >> Are you running a 32 or 64 bit JVM? >> Under 32 bit JVMs you could easily run out of available threads (the hig= her you tune your -Xmx, the LOWER the amount of available threads get!) >> >> >> Just a few random ideas... >> >> LieGrue, >> strub >> >> --- On Mon, 7/4/11, Martin Koci wrote: >> >>> From: Martin Koci >>> Subject: Re: MyFaces - Thread issues under High load >>> To: "MyFaces Discussion" >>> Date: Monday, July 4, 2011, 7:29 PM >>> Hi, >>> >>> what is [STUCK] thread? Is that something special from >>> WebLogic thread >>> pooler? >>> All threads are accessing same >>> java.beans.PropertyDescriptor.getReadMethod but this method >>> is >>> synchronized. So one on them is holding >>> java.beans.PropertyDescriptor >>> instance as monitor and the others are trying access same >>> critical area. >>> One of the STUCKs is suspicious: it probably entered >>> critical area on >>> PropertyDescriptor forever. But it should not happen: this >>> is API from >>> JRE and myfaces use it only - this looks very=A0 odd. >>> >>> Suggestions: >>> >>> 1) set thread timeout if possible. I don't know weblogic >>> but thread >>> executors can kill thread it it does not end in reasonable >>> time. >>> >>> 2) Is JVM eating 100% of CPU? If so then hotspot can cause >>> this type of >>> problem - I remement similar problem few year ago - update >>> to newer >>> version of JVM solved it. What JVM/version do you use? >>> >>> 3) Is problem repeatable? Do you perform stress test or is >>> this problem >>> from production deployment? >>> >>> >>> Regards, >>> >>> Ko=E8i=E8=E1k >>> >>> Rajadurai_p p=ED=B9e v Po 04. 07. 2011 v 11:17 -0700: >>> > Hi, >>> > I have attached a weblogic server thread dump file >>> collected under high load >>> > and when the server was irresponsive. You will find >>> almost all the threads >>> > (active & stuck) pointing to the same stack trace >>> which is My faces/ >>> > Tomahawk related. Kindly let me know your views. >>> > >>> > Thanks >>> > >>> > >>> > Matt Benson-2 wrote: >>> > > >>> > > It looks to me as though you are using the JSF >>> reference >>> > > implementation.=A0 The only code I see from >>> myfaces in your stack trace >>> > > is Tomahawk. >>> > > >>> > > Matt >>> > > >>> > > On Wed, Jun 29, 2011 at 4:39 AM, >>> Rajadurai_p=A0 wrote: >>> > >> >>> > >> Hi, >>> > >> we are facing problems in MyFaces framework >>> under high load. Bulk of the >>> > >> application container threads (weblogic) are >>> stuck upon a same stack >>> > >> trace. >>> > >> I have pasted a sample stack trace below. >>> Kindly help. >>> > >> >>> > >> We use JSF 1.2, MyFaces 1.2.9, Tomahawk 1.1.9 >>> & RichFaces 3.3.3. >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> java.beans.PropertyDescriptor.getReadMethod(PropertyDescriptor.java:158= ) >>> > >> >>> > >> >>> javax.faces.component._ComponentAttributesMap.getComponentProperty(_Com= ponentAttributesMap.java:382) >>> > >> >>> > >> >>> javax.faces.component._ComponentAttributesMap.get(_ComponentAttributesM= ap.java:227) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.html.HtmlRendererUtils.ren= derHTMLAttribute(HtmlRendererUtils.java:584) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.html.HtmlRendererUtils.ren= derHTMLAttributes(HtmlRendererUtils.java:598) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeBegin(HtmlTagRe= nderer.java:73) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeBegin(UIComponentBase.java:= 600) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:525) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:25= 8) >>> > >> >>> > >> >>> org.richfaces.renderkit.AbstractGridRenderer.encodeOneRow(AbstractGridR= enderer.java:96) >>> > >> >>> > >> >>> org.richfaces.renderkit.AbstractRowsRenderer.process(AbstractRowsRender= er.java:83) >>> > >> >>> > >> >>> org.ajax4jsf.model.SequenceDataModel.walk(SequenceDataModel.java:101) >>> > >> >>> > >> >>> org.ajax4jsf.component.UIDataAdaptorBase.walk(UIDataAdaptorBase.java:11= 56) >>> > >> >>> > >> >>> org.richfaces.renderkit.AbstractRowsRenderer.encodeRows(AbstractRowsRen= derer.java:104) >>> > >> >>> > >> >>> org.richfaces.renderkit.AbstractRowsRenderer.encodeRows(AbstractRowsRen= derer.java:88) >>> > >> >>> > >> >>> org.richfaces.renderkit.AbstractRowsRenderer.encodeChildren(AbstractRow= sRenderer.java:137) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:25= 8) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.html.AjaxOutputPanelRenderer.encodeChildren(Ajax= OutputPanelRenderer.java:78) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> javax.faces.component.UIComponent.encodeAll(UIComponent.java:250) >>> > >> >>> > >> >>> com.sun.facelets.component.RepeatRenderer.encodeChildren(RepeatRenderer= .java:64) >>> > >> >>> com.sun.facelets.component.UIRepeat.process(UIRepeat.java:423) >>> > >> >>> > >> >>> com.sun.facelets.component.UIRepeat.encodeChildren(UIRepeat.java:684) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:25= 8) >>> > >> >>> > >> >>> org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRender= er.java:200) >>> > >> >>> > >> >>> org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRender= er.java:195) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:12= 0) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:25= 8) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:279) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:25= 8) >>> > >> >>> > >> >>> org.ajax4jsf.renderkit.html.AjaxOutputPanelRenderer.encodeChildren(Ajax= OutputPanelRenderer.java:78) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:532) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChild(= RendererUtils.java:528) >>> > >> >>> > >> >>> org.apache.myfaces.shared_tomahawk.renderkit.RendererUtils.renderChildr= en(RendererUtils.java:511) >>> > >> >>> > >> >>> org.apache.myfaces.custom.htmlTag.HtmlTagRenderer.encodeChildren(HtmlTa= gRenderer.java:111) >>> > >> >>> > >> >>> javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.ja= va:627) >>> > >> >>> javax.faces.component.UIComponent.encodeAll(UIComponent.java:250) >>> > >> >>> javax.faces.component.UIComponent.encodeAll(UIComponent.java:257) >>> > >> >>> > >> >>> com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:= 594) >>> > >> >>> > >> >>> org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapp= er.java:100) >>> > >> >>> > >> >>> org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.jav= a:176) >>> > >> >>> > >> >>> org.apache.myfaces.lifecycle.RenderResponseExecutor.execute(RenderRespo= nseExecutor.java:41) >>> > >> >>> > >> >>> org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:14= 0) >>> > >> >>> javax.faces.webapp.FacesServlet.service(FacesServlet.java:187) >>> > >> >>> > >> >>> weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(S= tubSecurityHelper.java:227) >>> > >> >>> > >> >>> weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurity= Helper.java:125) >>> > >> >>> > >> >>> weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:= 283) >>> > >> >>> weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:206) >>> > >> >>> org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290) >>> > >> >>> > >> >>> org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilte= r.java:388) >>> > >> >>> org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.jav= a:164) >>> > >> >>> > >> >>> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141) >>> > >> >>> > >> >>> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewrite= r.java:90) >>> > >> >>> > >> >>> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteF= ilter.java:407) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFi= lter.java:384) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> com.xxx.yyy.zzz.CookieCartCountFilter.doFilter(CookieCartCountFilter.ja= va:69) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> com.xxx.yyy.zzz.webservice.onesite.OneSiteCacheFilter.doFilter(OneSiteC= acheFilter.java:388) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> com.xxx.yyy.zzz.util.CachingFilter.doFilter(CachingFilter.java:143) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> com.xxx.yyy.zzz.util.JsessionidFilter.doFilter(JsessionidFilter.java:43= ) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> com.xxx.yyy.zzz.util.SessionCookieSupportFilter.doFilter(SessionCookieS= upportFilter.java:180) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> de.hybris.platform.util.RootRequestFilter.doFilter(RootRequestFilter.ja= va:741) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFil= ter.java:26) >>> > >> >>> > >> >>> weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java= :42) >>> > >> >>> > >> >>> weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.= run(WebAppServletContext.java:3229) >>> > >> >>> > >> >>> weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedS= ubject.java:321) >>> > >> >>> > >> >>> weblogic.security.service.SecurityManager.runAs(SecurityManager.java:12= 1) >>> > >> >>> > >> >>> weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppSer= vletContext.java:2002) >>> > >> >>> > >> >>> weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletCon= text.java:1908) >>> > >> >>> > >> >>> weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.jav= a:1362) >>> > >> >>> weblogic.work.ExecuteThread.execute(ExecuteThread.java:209) >>> > >> >>> weblogic.work.ExecuteThread.run(ExecuteThread.java:181) >>> > >> >>> > >> Thanks, rajadurai >>> > >> -- >>> > >> View this message in context: >>> > >> http://old.nabble.com/MyFaces---Thread-issues-under-High-load-tp31= 953250p31953250.html >>> > >> Sent from the MyFaces - Users mailing list >>> archive at Nabble.com. >>> > >> >>> > >> >>> > > >>> > > >>> > http://old.nabble.com/file/p31991479/Weblogic%2BThread%2BDump.zip >>> > Weblogic+Thread+Dump.zip >>> > http://old.nabble.com/file/p31991479/Weblogic%2BThread%2BDump.zip >>> > Weblogic+Thread+Dump.zip >>> > -- >>> > View this message in context: http://old.nabble.com/MyFaces---Thread-= issues-under-High-load-tp31953250p31991479.html >>> > Sent from the MyFaces - Users mailing list archive at >>> Nabble.com. >>> >>> >>> >> >