lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
Date Mon, 19 Nov 2012 14:38:58 GMT


Shai Erera commented on LUCENE-4560:

bq. Small config changes shouldn't require the equivalent of a full optimize to take effect.

I wouldn't call these "small config changes" :). And if you give your users the possibility
to randomly decide to turn off tracking offsets ... well then either your users are all Lucene
highly-involved committers/users, or you're putting too much power in their hands.

I also can't tell, in your world - it's definitely not common in mines, how often will such
a configuration be changed. Removing fields no longer needed - that's one thing. Changing
IndexOptions, that's a totally different thing.

bq. Also, i argue that any addIndexes() approach is even more dangerous and just as prone
to corruption

Right, but because of how addIndexes works, you get the chance to review the result index
before you choose to publish it. If anything is not entirely as expected, you discard the
operation, fix the code, start over. The source index remains intact.
> Support Filtering Segments During Merge
> ---------------------------------------
>                 Key: LUCENE-4560
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Tim Smith
>         Attachments: LUCENE-4560.patch
> Spun off from LUCENE-4557
> It is desirable to be able to filter segments during merge.
> Most often, full reindex of content is not possible.
> Merging segments can sometimes have negative consequences when fields are have different
options (most restrictive option is forced during merge)
> Being able to filter segments during merges will allow gradually migrating indexed data
to new index settings, support pruning/enhancing existing data gradually
> Use Cases:
> * Migrate IndexOptions for fields (See LUCENE-4557)
> * Gradually Remove index fields no longer used
> * Migrate indexed sort fields to DocValues
> * Support converting data types for indexed data
> * and so on
> patch will be forthcoming

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message