lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Smith (JIRA)" <>
Subject [jira] Commented: (LUCENE-1703) Add a waitForMerges() method to IndexWriter
Date Fri, 19 Jun 2009 19:10:07 GMT


Tim Smith commented on LUCENE-1703:

I'm finding it a bit tricky to create a proper unit test for this

This is indirectly tested as IndexWriter.close() will call this method

but explicit calling of this method is harder to test as it requires outstanding merges to
be running (very difficult to trigger)

I started with TestIndexWriter.testNoWaitClose() as a base
however, modifying this to test waitForMerges() takes 11 seconds to run (surely because it
always waits, and doesn't abort the merges)

also, in order to properly test, another method should be added to IndexWriter
public synchronized int getOutstandingMerges() {
  return mergingSegments.size();

I suppose the most proper test would create a subclass of ConcurrentMergeScheduler that would
block all merges until after i released a lock.  Then the test would add docs, fork off thread
to unlock for merging in X seconds, then call waitForMerges.

> Add a waitForMerges() method to IndexWriter
> -------------------------------------------
>                 Key: LUCENE-1703
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.4
>            Reporter: Tim Smith
>         Attachments:
> It would be very useful to have a waitForMerges() method on the IndexWriter.
> Right now, the only way i can see to achieve this is to call IndexWriter.close()
> ideally, there would be a method on the IndexWriter to wait for merges without actually
closing the index.
> This would make it so that background merges (or optimize) can be waited for without
closing the IndexWriter, and then reopening a new IndexWriter
> the close() reopen IndexWriter method can be problematic if the close() fails as the
write lock won't be released
> this could then result in the following sequence:
> * close() - fails
> * force unlock the write lock (per close() documentation)
> * new IndexWriter() (acquires write lock)
> * finalize() on old IndexWriter releases the write lock
> * Index is now not locked, and another IndexWriter pointing to the same directory could
be opened
> If you don't force unlock the write lock, opening a new IndexWriter will fail until garbage
collection calls finalize() the old IndexWriter
> If the waitForMerges() method is available, i would likely never need to close() the
IndexWriter until right before the process being shutdown, so this issue would not occur (worst
case scenario, the waitForMerges() fails)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message