lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Will Johnson" <willjohnsonsea...@gmail.com>
Subject RE: Resource contention problem in Solrj
Date Tue, 18 Dec 2007 13:19:20 GMT
Fyi:  the CommonsHttpSolrServer already has method to do all of those
things:

  /** set connectionTimeout on the underlying
MultiThreadedHttpConnectionManager */
  public void setConnectionTimeout(int timeout) {
    _connectionManager.getParams().setConnectionTimeout(timeout);
  }
  
  /** set maxConnectionsPerHost on the underlying
MultiThreadedHttpConnectionManager */
  public void setDefaultMaxConnectionsPerHost(int connections) {
 
_connectionManager.getParams().setDefaultMaxConnectionsPerHost(connections);
  }
  
  /** set maxTotalConnection on the underlying
MultiThreadedHttpConnectionManager */
  public void setMaxTotalConnections(int connections) {
    _connectionManager.getParams().setMaxTotalConnections(connections);
  }


You can also get the underlying connection factory if you want to do other
crazier stuff.

  public MultiThreadedHttpConnectionManager getConnectionManager() {
    return _connectionManager;
  }



- will

-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Monday, December 17, 2007 9:18 PM
To: solr-dev@lucene.apache.org
Subject: Re: Resource contention problem in Solrj

Excellent!  Thanks for diagnosing this!
-Yonik

On Dec 17, 2007 9:00 PM, climbingrose <climbingrose@gmail.com> wrote:
> There seems to be resource contention problem with Solrj under load. To
> reproduce the problem: set up a sample webapp with solrj connect to a HTTP
> Solr instance and hammer the webapp with Apache ab (say 10 concurrent
> connection with 100 requests). You'll notice that the webapp's servlet
> container quickly consumes 100% CPU and stays there unless you restart it.
I
> can confirm that this happens with both Tomcat and Jetty. Meanwhile, the
> server that Solr is deployed on seems to be running fine.
>
> From this observation, I suspect that Solrj has connection contention
> problem. And this seems to be the case if you look at
CommonHttpSolrServer.
> This class uses MultiThreadedHttpConnectionManager which has
> maxConnectionsPerHost set to 2 by default. When the number of thread
> increases, this is obviously not enough and leads to connection contention
> problem. I quickly solve problem by adding another constructor to
> CommonHttpSolrServer that allows setting maxConnectionsPerHost and
> maxTotalConnections:
>
> public CommonsHttpSolrServer(int maxConsPerHost, int maxTotalCons, String
> solrServerUrl) throws MalformedURLException {
>     this(solrServerUrl);
>     this.maxConsPerHost = maxConsPerHost;
>     this.maxTotalCons = maxTotalCons;
>     HttpConnectionManagerParams params = new
HttpConnectionManagerParams();
>     params.setDefaultMaxConnectionsPerHost(maxConsPerHost);
>     params.setMaxTotalConnections(maxTotalCons);
>     _connectionManager.setParams(params);
> }
>
> Hope this information would help others.
>
> --
> Regards,
>
> Cuong Hoang
>


Mime
View raw message