lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Runo <mr...@zappos.com>
Subject Lucene sync bottleneck?
Date Thu, 26 Feb 2009 21:09:36 GMT
Hello folks!

I was under the impression that this sync bottleneck was fixed in  
recent versions of Solr/Lucene, but we're seeing it with 1.4-dev right  
now. When we load test a server with >100 threads (using jmeter), we  
see several threads all blocked at the same spot:

"http-8080-exec-505" - Thread t@594
    java.lang.Thread.State: BLOCKED on  
org.apache.lucene.index.SegmentReader@2b6f5d18 owned by: http-8080- 
exec-434
	at org.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java: 
737)
	at  
org 
.apache 
.lucene.index.MultiSegmentReader.isDeleted(MultiSegmentReader.java:266)
	at org.apache.solr.search.function.FunctionQuery 
$AllScorer.next(FunctionQuery.java:118)
	at org.apache.solr.search.function.FunctionQuery 
$AllScorer.skipTo(FunctionQuery.java:137)
	at  
org 
.apache 
.lucene 
.search.BooleanScorer2$SingleMatchScorer.skipTo(BooleanScorer2.java:170)
	at  
org.apache.lucene.search.ReqOptSumScorer.score(ReqOptSumScorer.java:76)
	at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java: 
357)
	at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java: 
320)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java: 
136)
	at org.apache.lucene.search.Searcher.search(Searcher.java:126)
	at org.apache.lucene.search.Searcher.search(Searcher.java:105)
	at  
org 
.apache 
.solr 
.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java: 
1231)
	at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:917)
	at  
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java: 
338)
	at  
org 
.apache 
.solr.handler.component.QueryComponent.process(QueryComponent.java:164)
	at  
org 
.apache 
.solr 
.handler.component.SearchHandler.handleRequestBody(SearchHandler.java: 
171)
	at  
org 
.apache 
.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java: 
131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333)
	at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
	at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
	at  
org 
.apache 
.catalina 
.core 
.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 
235)
	at  
org 
.apache 
.catalina 
.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at  
org 
.apache 
.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 
233)
	at  
org 
.apache 
.catalina.core.StandardContextValve.invoke(StandardContextValve.java: 
175)
	at  
org 
.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java: 
128)
	at  
org 
.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java: 
102)
	at  
org 
.apache 
.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at  
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 
286)
	at  
org 
.apache 
.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
	at org.apache.coyote.http11.Http11NioProtocol 
$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
	at org.apache.tomcat.util.net.NioEndpoint 
$SocketProcessor.run(NioEndpoint.java:2080)
	at java.util.concurrent.ThreadPoolExecutor 
$Worker.runTask(ThreadPoolExecutor.java:885)
	at java.util.concurrent.ThreadPoolExecutor 
$Worker.run(ThreadPoolExecutor.java:907)
	at java.lang.Thread.run(Thread.java:619)

    Locked ownable synchronizers:
	- locked java.util.concurrent.locks.ReentrantLock$NonfairSync@4d54c7be


I checked the Lucene SVN and it looks like that's still appearing to  
be a bottleneck.

http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java?view=markup

Any tips?

Thanks for your time!

Matthew Runo
Software Engineer, Zappos.com
mruno@zappos.com - 702-943-7833


Mime
View raw message