Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of ryantxu@gmail.com designates
 64.233.170.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding;
        b=ZtWziJLBsXKKxBlKyh0n8wxhRH+sA11quP4F8zTfwX3wNrM9j5pDflrhEjF8FgmubbiCIWpGgHwtWG+gbZ3AdfaVmzqYbpgT6kl9Xl1+5kndX0Tp43Rj4DidAC5tKZTs7mQMHSWabQ28ZsCjgGxX9TtO9QNlBSTtJiZcZCQdQ00=
Message-ID: <4767DE19.5050202@gmail.com>
Date: Tue, 18 Dec 2007 09:50:01 -0500
From: Ryan McKinley <ryantxu@gmail.com>
User-Agent: Thunderbird 2.0.0.9 (Windows/20071031)
MIME-Version: 1.0
To: solr-dev@lucene.apache.org
Subject: Re: Resource contention problem in Solrj
References: <3e7716cd0712171800y6366c5dcvab51d3eb9776abb4@mail.gmail.com>
In-Reply-To: <3e7716cd0712171800y6366c5dcvab51d3eb9776abb4@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

climbingrose wrote:
> There seems to be resource contention problem with Solrj under load. To
> reproduce the problem: set up a sample webapp with solrj connect to a HTTP
> Solr instance and hammer the webapp with Apache ab (say 10 concurrent
> connection with 100 requests). You'll notice that the webapp's servlet
> container quickly consumes 100% CPU and stays there unless you restart it. I
> can confirm that this happens with both Tomcat and Jetty. Meanwhile, the
> server that Solr is deployed on seems to be running fine.
> 
>>>From this observation, I suspect that Solrj has connection contention
> problem. And this seems to be the case if you look at CommonHttpSolrServer.
> This class uses MultiThreadedHttpConnectionManager which has
> maxConnectionsPerHost set to 2 by default. When the number of thread
> increases, this is obviously not enough and leads to connection contention

I'm reluctant to add a constructor to the API since all the params can 
be set via getConnectionManager()

perhaps we should increase the default maxConnectionsPerHost.  10?  We 
should also add some comment about setting it?

> 
> Hope this information would help others.
> 

yes, thanks!

ryan