lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "matt knecht (JIRA)" <>
Subject [jira] [Commented] (SOLR-4696) All threads become blocked resulting in hang when bulk adding
Date Wed, 10 Apr 2013 15:28:15 GMT


matt knecht commented on SOLR-4696:

Nagios shows that both solr nodes had a peak of about 50 open connections, 150 in TIME_WAIT
and maybe a dozen in CLOSE_WAIT.  Once the problem manifests, I see around 50 open, and 100
stuck in CLOSE_WAIT.  Both nodes look nearly identical.

The host doing the insert/updates shows a peak of around 100 open connections and around 9000
in TIME_WAIT.  Once the problem manifests all those TIME_WAIT connections clear.  During one
test, I gave tomcat a thread pool of 4096 and saw those eaten up as well.

On a successful load of all 350k documents, I see a similar pattern, perhaps a few less connections

> All threads become blocked resulting in hang when bulk adding
> -------------------------------------------------------------
>                 Key: SOLR-4696
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrCloud
>    Affects Versions: 4.1, 4.2, 4.2.1
>         Environment: Ubuntu 12.04.2 LTS 3.5.0-27-generic
> Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
> KVM, 4xCPU, 5GB RAM, 4GB heap.
> 4 cores, 2 shards, 2 nodes, tomcat7
>            Reporter: matt knecht
>              Labels: hang
>         Attachments: screenshot-1.jpg, solrconfig.xml, solr.jstack.1, solr.jstack.2
> During a bulk load after about 150,000 documents load, thread usage spikes, solr no longer
processes any documents.  Any additional documents added result in a new thread until the
pool is exhausted.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message