lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3082) Add index upgrade method to IndexWriter to force an upgrade of all segments to last recent supported index format without optimizing
Date Sun, 08 May 2011 18:22:03 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030518#comment-13030518
] 

Shai Erera commented on LUCENE-3082:
------------------------------------

This is a great idea. We should also allow one to plug in a PayloadProcessorProvider so he
can rewrite the payload "on the go" if need be.

Also, while the index is being upgraded, I think it will be useful if we merge the segments
that are upgraded, however not do cascading merges. Since segments are rewritten anyway, we
can only gain from the merge. As always, if not everybody agree on this, we can make it a
parameter.

And let's make sure that whatever 'upgrade' means is at the application control. I.e., upgrade
can be simply upgrading from 3x to 4.0, but it can also be using PayloadProcessorProvider
as well suddenly deciding that all segments should be compound. I'm pretty sure I'll want
to control the first two, not so about the last one.

It can be a simple 'boolean shouldUpgradeSegment(SegmentInfo)' on this UpgradeMP, which apps
can override.

> Add index upgrade method to IndexWriter to force an upgrade of all segments to last recent
supported index format without optimizing
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3082
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3082
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>            Reporter: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.2, 4.0
>
>
> Currently if you want to upgrade an old index to the format of your current Lucene version,
you have to optimize your index or use addIndexes(IndexReader...) [see LUCENE-2893] to copy
to a new directory. The optimize() approach fails if your index is already optimized.
> I propose to add a method to IndexWriter thats similar to optimize(), that uses a custom
MergePolicy to upgrade all segments to the last format. This MergePolicy could simply also
ignore all segments already up-to-date. All segments in prior formats would be merged to a
new segment. The tool could optionally also optimize the index.
> This issue is different from LUCENE-2893, as it would only support upgrading indexes
from previous Lucene versions in-place using the official path. Its a tool for the end user,
not a developer tool.
> This addition should also go to Lucene 3.x, as we need to make users with pre-3.0 indexes
go the step through 3.x, else they would not be able to open their index with 4.0. With this
tool in 3.x the users could safely upgrade their index without relying on optimize to work
on already-optimized indexes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message