lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1703) Add a waitForMerges() method to IndexWriter
Date Sat, 20 Jun 2009 10:42:07 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722168#action_12722168
] 

Michael McCandless commented on LUCENE-1703:
--------------------------------------------

bq. After I added a number of documents to the index some of the existing not-yet committed
segments get merged. It will never merge the current existing and committed segments, but
the segments created during indexing can be merged. After commit() or close() the already
existing and the new segments are maybe additionally merged and a new segments file written.

Actually, existing segments do get merged (assuming the merge policy selects them).  It's
just that they cannot be deleted because logically there are two snapshots while an IndexWriter
is running with autoCommit false: the index as of when it was opened, and the current in-memory
SegmentInfos.  (And if you are using a deletion policy different from the default KeepOnlyLastCommit,
even more snapshots will be kept).

> Add a waitForMerges() method to IndexWriter
> -------------------------------------------
>
>                 Key: LUCENE-1703
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1703
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.4
>            Reporter: Tim Smith
>         Attachments: IndexWriter.java.diff, IndexWriter.java.diff
>
>
> It would be very useful to have a waitForMerges() method on the IndexWriter.
> Right now, the only way i can see to achieve this is to call IndexWriter.close()
> ideally, there would be a method on the IndexWriter to wait for merges without actually
closing the index.
> This would make it so that background merges (or optimize) can be waited for without
closing the IndexWriter, and then reopening a new IndexWriter
> the close() reopen IndexWriter method can be problematic if the close() fails as the
write lock won't be released
> this could then result in the following sequence:
> * close() - fails
> * force unlock the write lock (per close() documentation)
> * new IndexWriter() (acquires write lock)
> * finalize() on old IndexWriter releases the write lock
> * Index is now not locked, and another IndexWriter pointing to the same directory could
be opened
> If you don't force unlock the write lock, opening a new IndexWriter will fail until garbage
collection calls finalize() the old IndexWriter
> If the waitForMerges() method is available, i would likely never need to close() the
IndexWriter until right before the process being shutdown, so this issue would not occur (worst
case scenario, the waitForMerges() fails)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message