lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <>
Subject RE: issue with IndexUpgrader
Date Thu, 29 Jan 2015 13:33:06 GMT
Hi Ken,

the reason for this behaviour is the following:
Index upgrader does not really do any special stuff to upgrade the index, it just uses standard
APIs. It implements a custom UpgradeMergePolicy that just looks for index segments which are
in an older format when doing a forceMerge(1) [aka "optimize"]. All those index segments are
queued for merging. After merging, it will of course write a new segments file, because the
index changed and IndexWriter was closed.

What happens in your zero-segment case is simple: There are no segments at all, so no segments
are chosen for merging. So forceMerge(1) is a no-op and no new index metadata is written by
IndexWriter. Unfortunately there is no easy way around. One possibility would be to check,
if the index has zero segments and then add/remove a dummy document and commit. An alternative
is to use filesystem magic and replace the index, as you suggested.

You might open a bug report for Lucene so we can fix this interesting corner case in IndexUpgrader,
and we can fix it in later versions, but we cannot do this in 3.x anymore, because the development
has ended there.


Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen

> -----Original Message-----
> From: Ken []
> Sent: Wednesday, January 28, 2015 8:06 PM
> To:
> Subject: issue with IndexUpgrader
> Hi,
>   I'm trying to run the 3.6.2 and 4.7.2 IndexUpgrader operations on a set of
> prior version Lucene indexes and I'm running into trouble with some corner
> case indexes.
>   Some (unknown set) of these indexes are just placeholders, they have
> been created but no documents have been added to them yet so they have
> zero segments. The index directory contains only segments.gen and
> segments_1.
> (Maybe it's possible to get into the same state by deleting all the documents
> from a once populated index.)
>  The 3.6.2 IndexUpgrader makes zero changes to these indexes, leaving
> them at the old version.  When the 4.7.2 IndexUpgrader runs it crashes with
> a read past the EOF of the unmodified segments_1 file.
>  Other than physically detecting a 'zero segment' index by examining the file
> system and replacing that index with a new 4.7.2 version empty index is
> there anything I can do?
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message