lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-847) Factor merge policy out of IndexWriter
Date Tue, 07 Aug 2007 19:48:02 GMT


Michael McCandless commented on LUCENE-847:

> Looking at IW, with the new DocsWriter stuff, it looks like there
> isn't a segmentInfo object for the new segment at the time the
> predicate is being called. Would it be possible to make one?
> Something like DocumentsWriter#getDocStoreSegmentInfo() analogous to
> DocumentsWriter#getDocStoreSegment()? It could be an object just
> thrown away.

Hmmm: it looks like you're calling
"mergePolicy.useCompoundFile(segmentInfos,null)" twice: once inside
flushDocStores and immediately thereafter when flushDocStores returns
into the flush() code.  Maybe you should change flushDocStores to
return whether or not the flushed files are in compound format

Since flushDocStores() is flushing only the "doc store" index files
(stored fields & term vectors) it's not a real "segment" so it's a
somewhat forced fit to make a SegmentInfo in this case.  I guess we
could make a "largely empty" SegmentInfo and fill in what we can, but
that's somewhat dangerous (eg methods like files() would have to be
fixed to deal with this).

Maybe, instead, we could use one of the SegmentInfo instances from a
segment that refers to this doc store segment?  This would just mean
stepping through all SegmentInfo's and finding the first one (say)
whose docStoreSegment equals the one we are now flushing?  Still it
would be good to differentiate this case (creating compound file for
doc store files vs for a real segment) to MergePolicy somehow (maybe
add a boolean arg "isDocStore" or some such?).

> Factor merge policy out of IndexWriter
> --------------------------------------
>                 Key: LUCENE-847
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Steven Parkes
>            Assignee: Steven Parkes
>         Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, LUCENE-847.txt
> If we factor the merge policy out of IndexWriter, we can make it pluggable, making it
possible for apps to choose a custom merge policy and for easier experimenting with merge
policy variants.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message