lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Checksum and transactional safety for lucene indexes
Date Tue, 21 Sep 2010 04:53:50 GMT
If an index file is not completely written to disk, it never become 
available. Lucene has a file describing the current active index 
segments. It writes all new files to the disk, and changes the 
description file (segments.gen) only after that.

If the index files are corrupted, all bets are off. Usually the data 
structures are damaged and Lucene throws CorruptIndexExceptions, NPE or 
array out-of-bounds exceptions. There is no checksumming of the index 
files.

Lance

Pulkit Singhal wrote:
> Hello Everyone,
>
> What happens if:
> a) lucene index gets written half-way to the disk and then something goes wrong?
> b) the index gets corrupted on the file system?
>
> When we open that directory location again using FSDirectory implementations:
> a) Is there any provision for the code to clean out the previous file
> and start a new index file because the older one was corrupted and
> didn't match the checksum?
> b) Or can we check that the # of documents that can be found in the
> underlying index are now ZERO because they can't be parsed properly?
> How can we do this?
>
> - Pulkit
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>    

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message