From java-user-return-38839-apmail-lucene-java-user-archive=lucene.apache.org@lucene.apache.org Tue Mar 03 13:14:07 2009 Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 38992 invoked from network); 3 Mar 2009 13:14:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 3 Mar 2009 13:14:07 -0000 Received: (qmail 20183 invoked by uid 500); 3 Mar 2009 13:14:00 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 20154 invoked by uid 500); 3 Mar 2009 13:14:00 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 20136 invoked by uid 99); 3 Mar 2009 13:14:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2009 05:14:00 -0800 X-ASF-Spam-Status: No, hits=2.6 required=10.0 tests=DNS_FROM_OPENWHOIS,SPF_HELO_PASS,SPF_PASS,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lists@nabble.com designates 216.139.236.158 as permitted sender) Received: from [216.139.236.158] (HELO kuber.nabble.com) (216.139.236.158) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2009 13:13:51 +0000 Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1LeUR7-0001sT-VQ for java-user@lucene.apache.org; Tue, 03 Mar 2009 05:13:29 -0800 Message-ID: <22308385.post@talk.nabble.com> Date: Tue, 3 Mar 2009 05:13:29 -0800 (PST) From: damu_verse To: java-user@lucene.apache.org Subject: Tomcat Threads are BLOCKED after some time MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: damoder.reddy@gmail.com X-Virus-Checked: Checked by ClamAV on apache.org Hi All, We Have used Lucene as our Search Engine and all our applications are deployed onto tomcat and running with thread pool size of 200. Java Version - 1.6.0-rc Lucene Version - 2.3.2 Tomcat Version - 6.0.14 OS - Red Hat Enterprise Linux ES release 4 (Nahant Update 5) kernel - 2.6.9-55.0.2.ELsmp RAM - 4 GB Tomcat Memory - 1.5 GB Index Size - 2 GB After 10-12 hrs of tomcat running, tomcat becomes irresponsive. After doing core dump of tomcat process We observed that all tomcat threads are blocked (Thread-pool size-200). none of the tomcat threads are in runnable state. each thread at the time of the core dump is in BLOCKED state...The following are the stack trace of blocked. "MultiSearcher thread #3" daemon prio=10 tid=0x337ddc00 nid=0x4827 waiting for monitor entry [0x2f2f0000..0x2f2f0ea0] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235) - waiting to lock <0x45d49d88> (a org.apache.lucene.store.FSDirectory$FSIndexInput) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76) at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:63) at org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:123) at org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:154) at org.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:223) at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:217) at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:54) at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:668) at org.apache.lucene.search.ConstantScoreTermQuery$TermWeight.scorer(ConstantScoreTermQuery.java:63) at org.apache.lucene.search.VBooleanQuery$BooleanWeight.scorer(VBooleanQuery.java:276) at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:232) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:143) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124) at org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:250) "http-8080-194" daemon prio=10 tid=0x08927800 nid=0x128d waiting for monitor entry [0x2e188000..0x2e189e20] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235) - waiting to lock <0x45d49d88> (a org.apache.lucene.store.FSDirectory$FSIndexInput) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38) at org.apache.lucene.store.IndexInput.readVLong(IndexInput.java:96) at org.apache.lucene.index.MultiLevelSkipListReader.loadSkipLevels(MultiLevelSkipListReader.java:196) at org.apache.lucene.index.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:97) at org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs.java:164) at in.verse.search.query.spans.TermSpans.skipTo(TermSpans.java:85) at in.verse.search.query.spans.SpanScorer.skipTo(SpanScorer.java:70) at org.apache.lucene.search.VConjunctionScorer.doNext(VConjunctionScorer.java:78) at org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:71) at org.apache.lucene.search.VBooleanScorer2.next(VBooleanScorer2.java:456) at org.apache.lucene.search.VConjunctionScorer.init(VConjunctionScorer.java:136) at org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:65) at org.apache.lucene.search.VBooleanScorer2.score(VBooleanScorer2.java:412) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146) at org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:173) at org.apache.lucene.search.Searcher.search(Searcher.java:118) at org.apache.lucene.search.Searcher.search(Searcher.java:97) at VSearcher$VerticalSearcher$1.call(VSearcher.java:500) at VSearcher$VerticalSearcher$1.call(VSearcher.java:491) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1737) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:200) at in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:542) at in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:476) at in.verse.search.VSearcher$VerticalSearcher.search(VSearcher.java:384) at in.verse.search.VSearcher.search(VSearcher.java:187) at in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:166) at in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:255) at org.apache.jsp.jobs.mobile.airtel.xhtml.search_jsp._jspService(search_jsp.java:869) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:393) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:852) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:584) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1508) at java.lang.Thread.run(Thread.java:619) -- View this message in context: http://www.nabble.com/Tomcat-Threads-are-BLOCKED-after-some-time-tp22308385p22308385.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org