lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "matt knecht (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4696) All threads become blocked resulting in hang when bulk adding
Date Wed, 10 Apr 2013 15:28:15 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627906#comment-13627906
] 

matt knecht commented on SOLR-4696:
-----------------------------------

Nagios shows that both solr nodes had a peak of about 50 open connections, 150 in TIME_WAIT
and maybe a dozen in CLOSE_WAIT.  Once the problem manifests, I see around 50 open, and 100
stuck in CLOSE_WAIT.  Both nodes look nearly identical.

The host doing the insert/updates shows a peak of around 100 open connections and around 9000
in TIME_WAIT.  Once the problem manifests all those TIME_WAIT connections clear.  During one
test, I gave tomcat a thread pool of 4096 and saw those eaten up as well.

On a successful load of all 350k documents, I see a similar pattern, perhaps a few less connections
in TIME_WAIT.


                
> All threads become blocked resulting in hang when bulk adding
> -------------------------------------------------------------
>
>                 Key: SOLR-4696
>                 URL: https://issues.apache.org/jira/browse/SOLR-4696
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrCloud
>    Affects Versions: 4.1, 4.2, 4.2.1
>         Environment: Ubuntu 12.04.2 LTS 3.5.0-27-generic
> Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
> KVM, 4xCPU, 5GB RAM, 4GB heap.
> 4 cores, 2 shards, 2 nodes, tomcat7
>            Reporter: matt knecht
>              Labels: hang
>         Attachments: screenshot-1.jpg, solrconfig.xml, solr.jstack.1, solr.jstack.2
>
>
> During a bulk load after about 150,000 documents load, thread usage spikes, solr no longer
processes any documents.  Any additional documents added result in a new thread until the
pool is exhausted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message