lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Server is shutting down due to threads
Date Thu, 04 Sep 2014 08:11:29 GMT
On 9/3/2014 3:31 PM, Ethan wrote:
> We have SolrCloud instance with 2 solr nodes and 3 zk ensemble.  One of the
> solr node goes down as soon as we send search traffic to it, but update
> works fine.
>
> When I analyzed thread dump I saw lot of blocked threads with following
> error message.  This explains why it couldn't create any native threads and
> ran out of memory.  The thread count went from 48 to 900 within minutes and
> server came down.  The other node with same configuration is taking all the
> search and update traffic, and it running fine.
>
> Any pointers would be appreciated.
>
> http-bio-52158-exec-59" - Thread t@589
>    java.lang.Thread.State: BLOCKED on
> org.apache.lucene.search.FieldCache$CreationPlaceholder@29e0400b owned by:
> http-bio-52158-exec-61

This may be completely useless information, but just in case, I thought
I'd throw it out there as something you should be able to quickly and
easily try.  I would also be interested in seeing what comes of Erick's
suggestions and questions.

The default maxThreads value on tomcat (which from your stacktrace is
very likely the container you're using) is 200.  That setting has been
increased to 10000 in the example jetty container that comes with Solr. 
Try setting maxThreads to 10000 in the configuration on all your tomcat
instances.  I *think* it would go in the HTTP Connector, but Jetty seems
to have it in the Server.

http://tomcat.apache.org/tomcat-5.5-doc/config/http.html

Thanks,
Shawn


Mime
View raw message