lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Serious Index Corruption Error - FileNotFoundException
Date Thu, 08 May 2008 07:55:25 GMT

OK, that sounds like a legitimate reason to forcibly remove the write  
lock, but it would be better to do that only on startup of your  
process rather than in every openIndex() call.

If ever you hit LockObtainFailedException in openIndex, even after  
having deleted the write lock on startup, then that means there's a  
bug (ie, two writers are in fact trying to open on the same index,  
one of the two cases below).

Mike

Jamie wrote:
> Hi Mike
>
> Thanks for the suggestions. I've implemented all of them. The main  
> reason why I manually deleted the lock file was that sometimes  
> users kill the server process manually or there is a hard reboot  
> without any warning. In such circumstances, Lucene leaves a lock  
> file lying around as it was busy writing to the index. Now, I  
> understand that one shouldn't simply delete the lock file, but what  
> do you suggest my users do? The server must continue running... the  
> only way that I see how is to delete the lock file, unless there is  
> the equivalent of chkdsk for Lucene indexes that I could run.
>
> Regards,
>
> Jmaie
>
>
> Michael McCandless wrote:
>>
>> On quickly looking through the code I think there are some serious  
>> hazards that could lead to this exception.
>>
>> First, in your openIndex code, if you hit a  
>> LockObtainFailedException in trying to open your writer, you are  
>> forcefully removing the write lock and then retrying.  Yet, you  
>> also open an IndexReader to delete documents, which acquires the  
>> write lock.  If ever you have this IndexReader open, and then you  
>> forcefully remove the write lock and open the writer, that would  
>> cause this exception.
>>
>> Second, you have a deletIndex method, which first tries to use the  
>> writer with create=true (good) but then falls back to manually  
>> removing the files.  Why is that fallback necessary?  If, for  
>> example, you are also hitting a LockObtainFailedException, then  
>> forcefully removing files while an IndexReader or IndexWriter  
>> holds the write lock is also dangerous and would lead to this  
>> exception.
>>
>> In general it's very dangerous to forcibly remove, or ignore,  
>> Lucene's write lock.  It really should only be necessary when  
>> something catastrophic occurred (JVM crashed).
>>
>> Also, note that IndexWriter can now delete documents.  This would  
>> simplify your code and possibly fix these two hazards.
>>
>> Do you see any of the error/warnings that you send to your  
>> logger?  (They would be corroborating evidence here).
>>
>> Mike
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message