lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3082) Add index upgrade method to IndexWriter to force an upgrade of all segments to last recent supported index format without optimizing
Date Sun, 08 May 2011 18:48:03 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030528#comment-13030528
] 

Michael McCandless commented on LUCENE-3082:
--------------------------------------------

Patch looks great!

The segmentsToOptimize ought to contain every segment in the index; that's only present for
the case where optimize() is called in a bg thread but other threads continue to index new
documents causing new segments to be flushed.  These new segments would then NOT be in the
segmentsToOptimize when the optimize merges need to cascade.

TODO: for the command-line tool, we should make sure the index only has a single commit point
(ie, abort if not).  Upgrading an index with more than one commit point is hairy (I think
it's fine not to support this case... but we should not remove the commits).

> Add index upgrade method to IndexWriter to force an upgrade of all segments to last recent
supported index format without optimizing
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3082
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3082
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>            Reporter: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.2, 4.0
>
>         Attachments: LUCENE-3082.patch, LUCENE-3082.patch, index.31.optimized.cfs.zip,
index.31.optimized.nocfs.zip
>
>
> Currently if you want to upgrade an old index to the format of your current Lucene version,
you have to optimize your index or use addIndexes(IndexReader...) [see LUCENE-2893] to copy
to a new directory. The optimize() approach fails if your index is already optimized.
> I propose to add a method to IndexWriter thats similar to optimize(), that uses a custom
MergePolicy to upgrade all segments to the last format. This MergePolicy could simply also
ignore all segments already up-to-date. All segments in prior formats would be merged to a
new segment. The tool could optionally also optimize the index.
> This issue is different from LUCENE-2893, as it would only support upgrading indexes
from previous Lucene versions in-place using the official path. Its a tool for the end user,
not a developer tool.
> This addition should also go to Lucene 3.x, as we need to make users with pre-3.0 indexes
go the step through 3.x, else they would not be able to open their index with 4.0. With this
tool in 3.x the users could safely upgrade their index without relying on optimize to work
on already-optimized indexes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message