lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-485) IndexWriter.mergeSegments should not hold the commit lock while cleaning up.
Date Wed, 15 Mar 2006 18:43:02 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-485?page=comments#action_12370563 ] 

Doug Cutting commented on LUCENE-485:
-------------------------------------

+1 This looks reasonable to me.  What is important is that IndexReader holds the COMMIT lock
while all files are opened.  But the writer does not need to be held while they're removed,
only while the set of files is changing, to keep readers from reading the set, then trying
to open files after they've been deleted.  This is still blocked after your patch, since the
set cannot change while files are being opened.

> IndexWriter.mergeSegments should not hold the commit lock while cleaning up.
> ----------------------------------------------------------------------------
>
>          Key: LUCENE-485
>          URL: http://issues.apache.org/jira/browse/LUCENE-485
>      Project: Lucene - Java
>         Type: Improvement
>   Components: Index
>     Versions: CVS Nightly - Specify date in submission
>  Environment: checked in revision 367361
>     Reporter: Luc Vanlerberghe
>  Attachments: LUCENE-485.patch
>
> Same happens in IndexWriter.addIndexes(IndexReader[] readers).
> The commit lock should be obtained whenever the Index structure/version is read or written.
 It should be kept for as short a period as possible.
> The write lock is needed to make sure only one IndexWriter or IndexReader instance can
update the index (multiple IndexReaders can of course use the index for searching).
> The list of files that can be deleted is stored in the file "deletable".  It is only
read or written by the IndexWriter instance that holds the write lock, so there's no need
to have the commit lock to to update it.
> On my production system deleting the obsolete segment files after a mergeSegments() happens
can occasionally take several seconds(!) and the commit lock blocks the searcher machines
from updating their IndexReader instance.
> Even on a standalone machine, the time to update the segments file is about 3ms, the
time to delete the obsolete segments about 30ms.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message