lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Parkes (JIRA)" <>
Subject [jira] Commented: (LUCENE-847) Factor merge policy out of IndexWriter
Date Tue, 07 Aug 2007 15:00:00 GMT


Steven Parkes commented on LUCENE-847:

	think we ideally would like concurrency to be fully independent of the
	merge policy.

I thought of that, too, while taking a fresh look at things again. It's my current approach,
though I'm not yet sure there won't be stumbling blocks. More soon, hopefully.

	I think with one change to your MergePolicy API & control flow, we
	could make this work very well: instead of requiring the MergePolicy
	to call IndexWriter.merge, and do the cascading, it should just return
	the one MergeSpecification that should be done right now.

Hmm ... interesting idea. I thought about it briefly, though I didn't pursue it (see below).
It would end up changing the possible space of merge policies subtly. You 	wouldn't be able
to have any state in the algorithm. Arguably this is a good thing. There is also a bit more
overhead, since starting the computation of potential merges from scratch each time could
imply a little more computation, but I suspect this is not significant.
	When the inner MergePolicy wants
	to do a merge, the ConcurrentMergePolicy would in turn kick off that
	merge in the BG but then return null to the IndexWriter allowing
	IndexWriter to return to its caller, etc.

I'm a little unsure here. Are you saying the ConcurrentMergePolicy does the merges itself,
rather than using the writer? That's going to mean a synchronization dance between the CMP
and the writer. There's no question but that there has to be some synch dance, but my current
thinking was to try to keep as cleanly within one class, IW, as I could.

> Factor merge policy out of IndexWriter
> --------------------------------------
>                 Key: LUCENE-847
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Steven Parkes
>            Assignee: Steven Parkes
>         Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, LUCENE-847.txt
> If we factor the merge policy out of IndexWriter, we can make it pluggable, making it
possible for apps to choose a custom merge policy and for easier experimenting with merge
policy variants.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message