lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Updated: (LUCENE-2770) Optimize SegmentMerger to work on atomic (Segment)Readers where possible
Date Fri, 19 Nov 2010 18:18:13 GMT


Uwe Schindler updated LUCENE-2770:

    Attachment: LUCENE-2770-3x.patch

Final patches, will commit soon.

The changes in MultiPassIndexSplitter are not yet really needed, but this is because FilterIndexReader
is broken in trunk (see related issue). But I add it here, because for consistency we need

This patch also removes the useless closeReaders() method in SegmentMerger. This was used
by tests only (incorrectly) and should never be called from production code.

> Optimize SegmentMerger to work on atomic (Segment)Readers where possible
> ------------------------------------------------------------------------
>                 Key: LUCENE-2770
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1, 4.0
>         Attachments: LUCENE-2770-3x.patch, LUCENE-2770-3x.patch, LUCENE-2770.patch, LUCENE-2770.patch,
> This is a spin-off from LUCENE-2769:
> Currently SegmentMerger has some optimizations when it merges segments that are SegmentReaders
(e.g. when doing normal indexing or optimizing). But when you do IndexWriter.addIndexes(IndexReader...)
the listed IndexReaders may not really be per-segment. SegmentMerger should track down all
passed in reads down to the lowest level (Segment)Reader (or other atomic readers like SlowMultiReaderWrapper)
and then merge. We can then remove most MultiFields usage (except term merging itsself) and
clean up the code.
> This especially saves lots of memory for merging norms, as no longer the duplicate norms
arrays are created when MultiReaders are used!

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message