lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Chitturi <>
Subject Re: SolrJ Socket Leak
Date Mon, 17 Feb 2014 08:03:55 GMT

I faced a similar issue when using CloudSolrServer with Solr. As Shawn
pointed out the 'TIME_WAIT' status happens when the connection is closed
by the http client. HTTP client closes connection whenever it thinks the
connection is stale
#d5e405). Even the docs point out the stale connection checking cannot be
all reliable. 

I see two ways to get around this:

	1. Enable 'SO_REUSEADDR'
	2. Disable stale connection checks.

Also by default, when we create CSS it does not explicitly configure any
http client parameters
pache/solr/client/solrj/impl/ In this case, the
default configuration parameters (max connections, max connections per
host) are used for a http connection. You can explicitly configure these
params when creating CSS using HttpClientUtil:

	ModifiableSolrParams params = new ModifiableSolrParams();
	params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 128);
	params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 32);
	params.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
	params.set(HttpClientUtil.PROP_CONNECTION_TIMEOUT, 30000);
	httpClient = HttpClientUtil.createClient(params);

	final HttpClient client = HttpClientUtil.createClient(params);
	LBHttpSolrServer lb = new LBHttpSolrServer(client);
	CloudSolrServer server = new CloudSolrServer(zkConnect, lb);

Currently, I am using http client 4.3.2 and building the client when
creating the CSS. I also use 'SO_REUSEADDR' option and I haven't seen the
'TIME_WAIT'  after this (may be because of better handling of stale
connections in 4.3.2 or because of 'SO_REUSEADDR' param enabled). My
current http client code looks like this: (works only with http client

	HttpClientBuilder httpBuilder = HttpClientBuilder.create();
	Builder socketConfig =  SocketConfig.custom();
	LBHttpSolrServer lb = new LBHttpSolrServer(httpClient, parser)
	CloudSolrServer server = new CloudSolrServer(zkConnect, lb);

There should be a way to configure socket reuse with 4.2.3 too. You can
try different configurations. I am surprised you have 'TIME_WAIT'
connections even after 30 minutes because 'TIME_WAIT' connection should be
closed by default in 2 mins by O.S I think.


Kiran Chitturi,

On 2/13/14 12:38 PM, "Jared Rodriguez" <> wrote:

>I am using solr/solrj 4.6.1 along with the apache httpclient 4.3.2 as part
>of a web application which connects to the solr server via solrj
>using CloudSolrServer();  The web application is wired up with Guice, and
>there is a single instance of the CloudSolrServer class used by all
>requests.  All this is running on Amazon.
>Basically, everything looks and runs fine for a while, but even with
>moderate concurrency, solrj starts leaving sockets open.  We are handling
>only about 250 connections to the web app per minute and each of these
>issues from 3 - 7 requests to solr.  Over a 30 minute period of this type
>of use, we end up with many 1000s of lingering sockets.  I can see these
>when running netstats
>tcp        0      0
>All to the same target host, which is my solr server. There are no other
>pieces of infrastructure on that box, just solr.  Eventually, the server
>just dies as no further sockets can be opened and the opened ones are not
>The solr server itself is unphased and running like a champ.  Average
>per request of 0.126, as seen in the solr web app admin UI query handler
>Apache httpclient had a bunch of leakage from version 4.2.x that they
>cleaned up and refactored in 4.3.x, which is why I upgraded.  Currently,
>solrj makes use of the old leaky 4.2 classes for establishing connections
>and using a connection pool.
>Jared Rodriguez

View raw message