lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Thessel <mich...@gossamer-threads.com>
Subject Re: Processor load
Date Tue, 03 Jul 2007 22:30:30 GMT
Hello UG

I couldn't solve the processor load problem. I decided to switch to
release 1.2. I currently run some tests (indexing 9 Million documents
and sending different search queries as fast as possible while indexing)
on the server and got some problems:

------------------------------------------
 208973 SEVERE: Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@75ca81e8:java
 208974     at
org.apache.lucene.search.ConjunctionScorer.init(ConjunctionScorer.java:97)
 208975     at
org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:54)
 208976     at
org.apache.lucene.search.BooleanScorer2.next(BooleanScorer2.java:338)
 208977     at org.apache.lucene.search.BooleanScorer
$SubScorer.<init>(BooleanScorer.java:56)
 208978     at
org.apache.lucene.search.BooleanScorer.add(BooleanScorer.java:84)
 208979     at
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:294)
 208980     at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
 208981     at
org.apache.lucene.search.Searcher.search(Searcher.java:118)
 208982     at
org.apache.lucene.search.Searcher.search(Searcher.java:97)
 208983     at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:888)
 208984     at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:805)
 208985     at org.apache.solr.search.SolrIndexSearcher.access
$100(SolrIndexSearcher.java:60)
 208986     at org.apache.solr.search.SolrIndexSearcher
$2.regenerateItem(SolrIndexSearcher.java:251)
 208987     at org.apache.solr.search.LRUCache.warm(LRUCache.java:193)
 208988     at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1385)
 208989     at org.apache.solr.core.SolrCore$1.call(SolrCore.java:488)
 208990     at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:269)
 208991     at java.util.concurrent.FutureTask.run(FutureTask.java:123)
 208992     at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:650)
 208993     at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:675)
 208994     at java.lang.Thread.run(Thread.java:595)
------------------------------------------


After this error message repeated for about 100 times I get this error:


------------------------------------------
SEVERE: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
 210622     at java.util.concurrent.FutureTask
$Sync.innerGet(FutureTask.java:205)
 210623     at java.util.concurrent.FutureTask.get(FutureTask.java:80)
 210624     at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:542)
 210625     at org.apache.solr.update.DirectUpdateHandler2
$CommitTracker.run(DirectUpdateHandler2.ja
 210626     at java.util.concurrent.Executors
$RunnableAdapter.call(Executors.java:417)
 210627     at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:269)
 210628     at java.util.concurrent.FutureTask.run(FutureTask.java:123)
 210629     at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.access$301(Sched
 210630     at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.run(ScheduledThr
 210631     at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:650)
 210632     at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:675)
 210633     at java.lang.Thread.run(Thread.java:595)
 210634 Caused by: java.lang.NullPointerException
 210635     at
org.apache.solr.core.SolrException.log(SolrException.java:67)
 210636     at
org.apache.solr.core.SolrException.logOnce(SolrException.java:85)
 210637     at org.apache.solr.core.SolrCore$4.call(SolrCore.java:546)
 210638     at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:269)
 210639     at java.util.concurrent.FutureTask.run(FutureTask.java:123)
 210640     ... 3 more
------------------------------------------

And finally it ends up with a LockObtainFailedException:

------------------------------------------
SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain
timed out: SimpleFSLo
 211054     at org.apache.lucene.store.Lock.obtain(Lock.java:70)
 211055     at
org.apache.lucene.index.IndexWriter.init(IndexWriter.java:579)
 211056     at
org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:341)
 211057     at
org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:65)
 211058     at
org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:120)
 211059     at
org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:181)
 211060     at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:259)
 211061     at
org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandler.java:1
 211062     at
org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHan
 211063     at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
 211064     at org.apache.solr.core.SolrCore.execute(SolrCore.java:658)
 211065     at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191)
 211066     at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159)
 211067     at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterCha
 211068     at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
 211069     at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228)
 211070     at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
 211071     at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
 211072     at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
 211073     at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 211074     at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:216)
 211075     at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
 211076     at org.apache.coyote.http11.Http11Protocol
$Http11ConnectionHandler.process(Http11Protoco
 211077     at org.apache.tomcat.util.net.JIoEndpoint
$Worker.run(JIoEndpoint.java:445)
 211078     at java.lang.Thread.run(Thread.java:595)
------------------------------------------

The first two errors only appear when I send queries while I'm indexing.
The LockObtainFailedException always comes up after 1-2 hours of
indexing.

I found this patch for the LockObtainFailedException problem. Is this
currently the official way of fixing the problem? Is this patch already
available with one of the nightly builds?

https://issues.apache.org/jira/browse/SOLR-240


Thanks and Cheers,

Michael



On Thu, 2007-28-06 at 16:31 -0700, Otis Gospodnetic wrote:
> I think you'll need to add this to your Java command line (but you'll need to kill your
JVM first):
>      -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9889 ....
> 
> Otis
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share
> 
> ----- Original Message ----
> From: Mike Klaas <mike.klaas@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Friday, June 29, 2007 1:01:03 AM
> Subject: Re: Processor load
> 
> On 28-Jun-07, at 10:52 AM, Michael Thessel wrote:
> 
> > Hi Mike
> >
> > Thanks for your quick response.
> > The restarting the server hasn't any effect. Im not a Java expert and
> > not really familiar with the java tools.
> 
> I'm no expert in this matter either.  Have you tried kill -QUIT? (it  
> prints a thread dump to stderr).
> 
> -Mike
> 
> > Running jconsole on the 100% process (as the same user as the  
> > server is
> > running):
> > # jconsole 23248
> > Unable to attach to 23248: Could not map vmid to user Name
> >
> > Running jps:
> > # jps -mlv
> > 11824 sun.tools.jps.Jps -mlv
> > -Dapplication.home=/var/home/solr/src/jdk1.5.0_11 -Xms8m
> > 23228 org.apache.catalina.startup.Bootstrap start -Xms1024M -Xmx2048M
> > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > -Djava.util.logging.config.file=/home/solr/tomcat/conf/ 
> > logging.properties -Djava.endorsed.dirs=/home/solr/tomcat/endorsed - 
> > Dcatalina.base=/home/solr/tomcat -Dcatalina.home=/home/solr/tomcat - 
> > Djava.io.tmpdir=/home/solr/tomcat/temp
> >
> > Running jconsole on the parent process:
> > # jconsole 23228
> > 23228 is not a managed VM.
> >
> >
> > Cheers,
> >
> > Michael
> >
> >
> > On Thu, 2007-28-06 at 10:14 -0700, Mike Klaas wrote:
> >> On 28-Jun-07, at 9:36 AM, Michael Thessel wrote:
> >>
> >>> Hello UG
> >>>
> >>> I got a solr server running to index the posts of a forum. The
> >>> server is
> >>> running in a tomcat installation on a 4-processor Opteron server.
> >>> One of
> >>> the threads is eating up 100% of one processor all the time. As you
> >>> can
> >>> see in the ps output it is a fork of one of the childs:
> >>
> >> have you tried looking at a thread dump using jConsole/kill -QUIT?
> >>
> >> -Mike
> > -- 
> > Michael Thessel <michael@gossamer-threads.com>
> > Gossamer Threads Inc. http://www.gossamer-threads.com/
> > Tel: (604) 687-5804 Fax: (604) 687-5806
> >
-- 
Michael Thessel <michael@gossamer-threads.com>
Gossamer Threads Inc. http://www.gossamer-threads.com/
Tel: (604) 687-5804 Fax: (604) 687-5806


Mime
View raw message