lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt
Date Thu, 29 Jul 2010 10:32:36 GMT
On Wed, Jul 28, 2010 at 1:08 AM, David Sitsky <sits@nuix.com> wrote:
> Incidentally.. this is what TerminateProcess does say from MSDN:
>
> TerminateProcess initiates termination and returns immediately. This
> stops execution of all threads within the process and requests
> cancellation of all pending I/O. The terminated process cannot exit
> until all pending I/O has been completed or canceled.

Hmm that language is rather spooky -- the cancellation of all "pending
I/O" sounds dangerous.  So the question is what Windows API the JRE
uses for fsync, and whether Windows considers that "pending I/O".

It seems unlikely that an fsync, once returned, can be cancelled by
the OS, since the OS must forward the request to the IO system, wait
for it to move the bytes to stable storage (write cache in your case),
before returning.

> So I am not sure if this is a problem or not.. if a background merge
> is happening while TerminateProcess() is called, then potentially, the
> Lucene libraries may have written some data, but I wonder if the O/S
> can "cancel" it before it is committed?  You would hope if fsync()
> equivalent is called, that the pending I/O operations can't be
> cancelled, but I do wonder?

I do wonder as well :)  But I can't imagine Windows can cancel already
completed fsync's on TerminateProcess.

Any bytes written by Lucene but not fsync'd are perfectly fair game to
"lose" -- that won't affect the index because we do not remove the old
commit point until all files referenced by the new commit point have
been fsync'd.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message