lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-847) Factor merge policy out of IndexWriter
Date Wed, 12 Sep 2007 08:56:32 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526712
] 

Michael McCandless commented on LUCENE-847:
-------------------------------------------

> > Good idea! I took exactly this approach in patch I just attached. I
> > made a simple change: LogMergePolicy.findMergesForOptimize first
> > checks if "normal merging" would want to do merges and returns them if
> > so. Since "normal merging" exposes concurrent merges, this gains us
> > concurrency for optimize in cases where the index has too many
> > segments. I wasn't sure how otherwise to expose concurrency...
>
> Another option is to schedule merges for the newest N segments and
> the next newest N segments and the next next... N is the merge
> factor.

OK, that is simpler.  I'll take that approach (and not call the
"normal" merge policy first).

> A couple of other things:
> 
>   - It seems you intended sync() to be part of the MergeScheduler
>     interface?

I had started down this route but then backed away from it: I think
IndexWriter should handle this rather than making every MergeScheduler
have duplicated code for doing so.  Oh I see, I had left empty sync()
in SerialMergeScheduler; I'll remove that.

>  - IndexWriter.close([true]), abort(): The behaviour should be the
>    same whether the calling thread is the one that actually gets to do
>    the closing. Right now, only the thread that actually does the
>    closing waits for the closing. The others do not wait for the
>    closing.

Ahh good point.  OK, I'll have other threads wait() until the
close/abort is complete.


> Factor merge policy out of IndexWriter
> --------------------------------------
>
>                 Key: LUCENE-847
>                 URL: https://issues.apache.org/jira/browse/LUCENE-847
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Steven Parkes
>            Assignee: Steven Parkes
>             Fix For: 2.3
>
>         Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, LUCENE-847.patch.txt,
LUCENE-847.take3.patch, LUCENE-847.take4.patch, LUCENE-847.take5.patch, LUCENE-847.take6.patch,
LUCENE-847.txt
>
>
> If we factor the merge policy out of IndexWriter, we can make it pluggable, making it
possible for apps to choose a custom merge policy and for easier experimenting with merge
policy variants.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message