lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Klaas <mike.kl...@gmail.com>
Subject Re: DirectUpdateHandler2 threads pile up behind scheduleCommitWithin
Date Tue, 12 May 2009 00:19:40 GMT
Hi Jayson,

Thanks, I'll take a look in the next few days.  The current patch  
doesn't guarantee index consistency during post-commit callback hooks,  
right?  This could be a problem for index replication.  (Incidentally,  
I'm rather unfamiliar with the new java-based replication design.   
Anyone care to comment on the implications?)

cheers,
-MIke

On 10-May-09, at 10:54 AM, jayson.minard wrote:

>
> Mike,
>
> I revamped the DirectUpdateHandler2 into DirectUpdateHandler3 in  
> SOLR-1155,
> probably ready enough for your review to see if locking makes sense  
> for
> current Lucene behavior.
>
> https://issues.apache.org/jira/browse/SOLR-1155
>
> --j
>
>
> Mike Klaas wrote:
>>
>> On 7-May-09, at 10:36 AM, jayson.minard wrote:
>>
>>>
>>> Does every thread really need to notify the update handler of the
>>> commit
>>> interval/threshold being reached, or really just the first thread  
>>> that
>>> notices should send the signal, or better yet a background commit
>>> watching
>>> thread so that no "foreground" thread has to pay attention at all.
>>> That is
>>> assuming they wouldn't need to block like they are now for a reason
>>> I'm
>>> likely unaware of...
>>
>> This is due to the way Lucene was designed (although recent
>> improvements in Lucene mean we can do better here).  See the recent
>> thread "Autocommit blocking adds?" on solr-user for a related
>> discussion.
>>
>> As the person who first wrote the multi-threaded-ness of DUH2, I'd be
>> very happy to promptly review any improvements made to it.
>>
>> -Mike
>>
>>
>
> -- 
> View this message in context: http://www.nabble.com/DirectUpdateHandler2-threads-pile-up-behind-scheduleCommitWithin-tp23431691p23472391.html
> Sent from the Solr - Dev mailing list archive at Nabble.com.
>


Mime
View raw message