lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: solr/tomcat stops responding
Date Fri, 01 Dec 2006 15:10:13 GMT
On 12/1/06, Kevin Lewandowski <kevinsl@gmail.com> wrote:
> My solr installation has been running fine for a few weeks but now
> after a server reboot it starts and runs for a few seconds, then stops
> responding. I don't see any errors in the logfiles, apart from
> snapinstaller not being able to issue a commit. Also, the process is
> using 100% cpu and stops responding to http requests (admin interface
> and queries).

- check what got changed after the server reboot... anything?
- turn on GC logging to see if you are running out of memory and the
GC system is taking up the CPU:  -Xloggc:gc.log
- get a JVM thread dump via kill -3 (assuming Linux/Unix)
- try rebuilding the index to see if it's something funky there

> Is there anything in solr that would cause this?

Yes, if it's normally a heavily loaded server that was restarted,
there can be a problem
of too many search requests hitting it all at once.  Many of the
requests may want to sort by the same field, and all of them start
generating the same fieldcache entry or filter entries at the same
time.  This causes a lot of memory to quickly be allocated, GC starts
churning trying to free up memory, more threads pile up, etc...

Part of the fix for this has recently been committed into Lucene
(multiple threads won't generate the same FieldCache entry).  We could
also do more in Solr by not releasing the first searcher until it has
been warmed by the static solrconfig.xml entries.

To see if this is your problem, restart the server and make sure no
traffic goes to it.
Then run some queries of the same type that will be hitting it to warm
it up, then turn on normal traffic.

-Yonik

Mime
View raw message