lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aakarsh Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1711) Race condition in org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.java
Date Tue, 08 Feb 2011 23:10:57 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992224#comment-12992224
] 

Aakarsh Nair commented on SOLR-1711:
------------------------------------

We are still seeing this issue even after using Johannes fix. All runners are exiting  and
the main producer thread hangs on line 196 queue.put . I am thinking it may be because queue
is getting drained and filled fast (queue size 50 , number of threads 20) . So there might
be a race condition on the queue capacity check.Queue appears to be below capacity to the
last runner  
then  fills up by simultaneous calls to put . I still see the issue after backporting  what
is in 3.x branch for testing it with solr 1.4.1. I guess a solution may be to use larger queue
capacities for now but the race conditions still seem to be present.

> Race condition in org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.java
> ----------------------------------------------------------------------------------
>
>                 Key: SOLR-1711
>                 URL: https://issues.apache.org/jira/browse/SOLR-1711
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 1.4, 1.5
>            Reporter: Attila Babo
>            Assignee: Yonik Seeley
>            Priority: Critical
>             Fix For: 1.4.1, 1.5, 3.1, 4.0
>
>         Attachments: StreamingUpdateSolrServer.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> While inserting a large pile of documents using StreamingUpdateSolrServer there is a
race condition as all Runner instances stop processing while the blocking queue is full. With
a high performance client this could happen quite often, there is no way to recover from it
at the client side.
> In StreamingUpdateSolrServer there is a BlockingQueue called queue to store UpdateRequests,
there are up to threadCount number of workers threads from StreamingUpdateSolrServer.Runner
to read that queue and push requests to a Solr instance. If at one point the BlockingQueue
is empty all workers stop processing it and pushing the collected content to Solr which could
be a time consuming process, sometimes all worker threads are waiting for Solr. If at this
time the client fills the BlockingQueue to full all worker threads will quit without processing
any further and the main thread will block forever.
> There is a simple, well tested patch attached to handle this situation.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message