lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-847) Factor merge policy out of IndexWriter
Date Wed, 12 Sep 2007 08:56:32 GMT


Michael McCandless commented on LUCENE-847:

> > Good idea! I took exactly this approach in patch I just attached. I
> > made a simple change: LogMergePolicy.findMergesForOptimize first
> > checks if "normal merging" would want to do merges and returns them if
> > so. Since "normal merging" exposes concurrent merges, this gains us
> > concurrency for optimize in cases where the index has too many
> > segments. I wasn't sure how otherwise to expose concurrency...
> Another option is to schedule merges for the newest N segments and
> the next newest N segments and the next next... N is the merge
> factor.

OK, that is simpler.  I'll take that approach (and not call the
"normal" merge policy first).

> A couple of other things:
>   - It seems you intended sync() to be part of the MergeScheduler
>     interface?

I had started down this route but then backed away from it: I think
IndexWriter should handle this rather than making every MergeScheduler
have duplicated code for doing so.  Oh I see, I had left empty sync()
in SerialMergeScheduler; I'll remove that.

>  - IndexWriter.close([true]), abort(): The behaviour should be the
>    same whether the calling thread is the one that actually gets to do
>    the closing. Right now, only the thread that actually does the
>    closing waits for the closing. The others do not wait for the
>    closing.

Ahh good point.  OK, I'll have other threads wait() until the
close/abort is complete.

> Factor merge policy out of IndexWriter
> --------------------------------------
>                 Key: LUCENE-847
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Steven Parkes
>            Assignee: Steven Parkes
>             Fix For: 2.3
>         Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, LUCENE-847.patch.txt,
LUCENE-847.take3.patch, LUCENE-847.take4.patch, LUCENE-847.take5.patch, LUCENE-847.take6.patch,
> If we factor the merge policy out of IndexWriter, we can make it pluggable, making it
possible for apps to choose a custom merge policy and for easier experimenting with merge
policy variants.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message