lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Updated: (LUCENE-2111) Wrapup flexible indexing
Date Tue, 09 Feb 2010 11:43:27 GMT


Michael McCandless updated LUCENE-2111:

    Attachment: LUCENE-2111.patch

New flex patch attached:

  * I factored out separate public Multi* (Fields, Terms, etc.) from
    DirectoryReader, DirectoryReader.  These classes merge multiple
    flex "sub readers" into a single flex API on the fly.

  * Refactored all places that need to merge sub-readers to use this
    API (DirectoryReader, MultiReader, SegmentMerger).  This is
    cleaner because previously SegmentMerger had its own duplicated
    code for doing this merging; now we have a single source for it
    (though merging swaps in its own docs/positions enum, to remap
    docIDs around deletions).

  * Changed the semantics of IndexReader.fields() -- for a multi
    reader (any reader that consist of sequential sub readers),
    fields() now throws UOE.

This is an important change with flex -- the caller now bears
responsibility for create a MultiFields if they really need it.

My thinking is that primary places in Lucene that consume postings now
operate per-segment, so a multi reader (Dir/MultiReader) should not
automatically "join up high" because it entails a hidden performance
hit.  So consumers that must access the flex API at the multi reader
level should be explicit about it...

However, to make this simple, I created a sugar static methods on
MultiFields (eg, MultiFields.getFields(IndexReader)) to easily do
this, and cutover places in Lucene that may need direct postings from
a multi-reader to use this method.

I've updated the javadocs explaining this.

> Wrapup flexible indexing
> ------------------------
>                 Key: LUCENE-2111
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: Flex Branch
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 3.1
>         Attachments: LUCENE-2111.patch, LUCENE-2111.patch, LUCENE-2111.patch, LUCENE-2111.patch,
LUCENE-2111.patch, LUCENE-2111.patch, LUCENE-2111.patch
> Spinoff from LUCENE-1458.
> The flex branch is in fairly good shape -- all tests pass, initial search performance
testing looks good, it survived several visits from the Unicode policeman ;)
> But it still has a number of nocommits, could use some more scrutiny especially on the
"emulate old API on flex index" and vice/versa code paths, and still needs some more performance
testing.  I'll do these under this issue, and we should open separate issues for other self
contained fixes.
> The end is in sight!

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message