lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johannes Goll <johannes.g...@gmail.com>
Subject Re: Huge performance drop in distributed search w/ shards on the same server/container
Date Tue, 14 Jun 2011 13:44:55 GMT
I increased the maximum POST size and headerBufferSize to 10MB ; lowThreads
to 50, maxThreads to 100000 and lowResourceMaxIdleTime=15000. We tried
tomcat 6 using the following Connnector settings :

<Connector port="8989"
       protocol="HTTP/1.1"
       redirectPort="8443"
       URIEncoding="UTF-8"
       maxThreads="10000"
       maxKeepAliveRequests="200"
       maxHttpHeaderSize="10485760"
       />

I am getting the same exception as for jetty

SEVERE: org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: java.net.SocketException:
Connection reset

This seem to point towards a Solr specific issue (solrj.SolrServerException
during individual shard searches).  I monitored the CPU utilization
executing sequential distributed searches and noticed that in the beginning
all CPUs are getting used for a short period of time (multiple lines for
shard searches are shown in the log with isShard=true arguments), then all
CPU except one become idle and the request is being processed by this one
CPU for the longest period of time.

I also noticed in the logs that while most of the individual shard searches
(isShard=true) have low QTimes (5-10), a minority has extreme QTimes
(104402-105126). All shards are fairly similar in size and content (1.2 M
documents) and the StatsComponent is being used
[stats=true&stats.field=weight&stats.facet=library_id]. Here library_id
equals the shard/core name.

Is there an internal timeout for gathering shard results or other fixed
resource limitation ?

Johannes






2011/6/13 Yonik Seeley <yonik@lucidimagination.com>

> On Sun, Jun 12, 2011 at 9:10 PM, Johannes Goll <johannes.goll@gmail.com>
> wrote:
> > However, sporadically, Jetty 6.1.2X (shipped with  Solr 3.1.)
> > sporadically throws Socket connect exceptions when executing distributed
> > searches.
>
> Are you using the exact jetty.xml that shipped with the solr example
> server,
> or did you make any modifications?
>
> -Yonik
> http://www.lucidimagination.com
>



-- 
Johannes Goll
211 Curry Ford Lane
Gaithersburg, Maryland 20878

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message