lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pulkit Singhal <>
Subject Re: Checksum and transactional safety for lucene indexes
Date Fri, 24 Sep 2010 14:57:27 GMT
In order to determine the integrity of an index file, I found that the
easiest way was to use and if there were
any problems with the data then catch the exceptions and make a new

I also see that the API offers IndexReader.indexExists() ... would
that be a better alternative?
Or would I miss out on being able to check if the index is corrupt if
I limit myself to that invocaction only?


- Pulkit

On Tue, Sep 21, 2010 at 12:53 AM, Lance Norskog <> wrote:
> If an index file is not completely written to disk, it never become
> available. Lucene has a file describing the current active index segments.
> It writes all new files to the disk, and changes the description file
> (segments.gen) only after that.
> If the index files are corrupted, all bets are off. Usually the data
> structures are damaged and Lucene throws CorruptIndexExceptions, NPE or
> array out-of-bounds exceptions. There is no checksumming of the index files.
> Lance
> Pulkit Singhal wrote:
>> Hello Everyone,
>> What happens if:
>> a) lucene index gets written half-way to the disk and then something goes
>> wrong?
>> b) the index gets corrupted on the file system?
>> When we open that directory location again using FSDirectory
>> implementations:
>> a) Is there any provision for the code to clean out the previous file
>> and start a new index file because the older one was corrupted and
>> didn't match the checksum?
>> b) Or can we check that the # of documents that can be found in the
>> underlying index are now ZERO because they can't be parsed properly?
>> How can we do this?
>> - Pulkit
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message