lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Smith (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
Date Mon, 19 Nov 2012 15:10:59 GMT


Tim Smith commented on LUCENE-4560:

bq. Its been this way since even 2.x, if you omitTF, then later decide you want TF and positions,
you need to re-index.

re-index is the key word here
re-indexing is not something that can always be done, or implies a massive cost.
changing a schema setting for one field should not require a full re-index.
i'm afraid i'm in a world where re-index is a 4 letter word and should only be done in the
most extreme of circumstances.
my whole point here is that migration should be possible via a pluggable policy

bq. Thats why these are expert options.

i know these are expert options, but there should also be a means to support migration to
new settings (albeit an expert means to do so that may have some consequences for how old
documents were indexed)

> Support Filtering Segments During Merge
> ---------------------------------------
>                 Key: LUCENE-4560
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Tim Smith
>         Attachments: LUCENE-4560.patch
> Spun off from LUCENE-4557
> It is desirable to be able to filter segments during merge.
> Most often, full reindex of content is not possible.
> Merging segments can sometimes have negative consequences when fields are have different
options (most restrictive option is forced during merge)
> Being able to filter segments during merges will allow gradually migrating indexed data
to new index settings, support pruning/enhancing existing data gradually
> Use Cases:
> * Migrate IndexOptions for fields (See LUCENE-4557)
> * Gradually Remove index fields no longer used
> * Migrate indexed sort fields to DocValues
> * Support converting data types for indexed data
> * and so on
> patch will be forthcoming

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message