lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4696) All threads become blocked resulting in hang when bulk adding
Date Wed, 10 Apr 2013 02:56:16 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627405#comment-13627405
] 

Erick Erickson commented on SOLR-4696:
--------------------------------------

Hmmm, I strongly suspect there's something you're leaving out. Lots of people are indexing
more than 150K docs, so what you're seeing must be something peculiar to your setup. Which
we need to understand....

So, how are you indexing documents? SolrJ? DIH? post.jar? Have you bounced any of your servers?
Because your autocommit interval is actually pretty long. Doing an <autoCommit> with
openSearcher=false is actually a pretty cheap operation that truncates your transaction log
and starts a new segment. The transaction log is replayed upon node startup. So if you're
bouncing a node, it's going to replay everything since the last hard commit.

So try lowering the autocommit time to, say, 15 seconds. This doesn't really explain the behavior
you're seeing especially if you're not bouncing your Solr servers, but it'd be interesting
information.
                
> All threads become blocked resulting in hang when bulk adding
> -------------------------------------------------------------
>
>                 Key: SOLR-4696
>                 URL: https://issues.apache.org/jira/browse/SOLR-4696
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrCloud
>    Affects Versions: 4.1, 4.2, 4.2.1
>         Environment: Ubuntu 12.04.2 LTS 3.5.0-27-generic
> Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
> KVM, 4xCPU, 5GB RAM, 4GB heap.
> 4 cores, 2 shards, 2 nodes, tomcat7
>            Reporter: matt knecht
>              Labels: hang
>         Attachments: screenshot-1.jpg, solrconfig.xml, solr.jstack.1, solr.jstack.2
>
>
> During a bulk load after about 150,000 documents load, thread usage spikes, solr no longer
processes any documents.  Any additional documents added result in a new thread until the
pool is exhausted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message