lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: After kill -9 index was corrupt
Date Mon, 11 Sep 2006 13:57:11 GMT
Yonik Seeley wrote:
> On 9/10/06, Chuck Williams <chuck@manawiz.com> wrote:
>> Could a kill -9 prevent data from reaching disk for files that were
>> previously closed?
> 
> No.  After a close() the OS should have all the data... the process
> may be killed but the OS will eventually flush all the buffers, etc.
> File creation is pretty much always synchronous so I have no idea how
> your problem could have happened (missing segment files).  IO error or
> something else temporarily filling up the disk?
> 
> If you have a power loss or crash, then that *can* cause data loss.
> There may be mount options to make more file operations synchronous,
> or you could maybe write your own Directory implementation to make
> things more synchronous.

Agreed ... it's hard to explain how this could have occurred without
the OS / machine actually going down abruptly.

Is this all on a single machine, local filesystem?

Are you really sure your underlying IO system is "healthy", no silent
file system corruption going on or anything?

However, I do think it would be a good idea to [optionally] add a
sync() call on committing the segments file to still be robust to OS /
machine crashing... it would slow down performance of indexing but
hopefully not by too much since the segments file is small.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message