lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <>
Subject Re: [jira] Resolved: (LUCENE-1044) Behavior on hard power shutdown
Date Sun, 04 Nov 2007 01:49:16 GMT
I think using FD.sync() might have enabled the proper operation on  
NFS in a much simpler way...

I argued quite a bit that the approach people were taking was  
probably not correct, and/or a bug in NFS (which it seems there was  
at least one that was corrected in a later release).

IMHO the work to make Lucene "work" on NFS was wasted - anyone using  
NFS (and that knows the design of it) would say that it was against  
the design.

They only needed to move Lucene into a "server" installation. Running  
Lucene over NFS just isn't a good use of bandwidth or the architecture.

You ended up making Lucene way more complex for a very boundary  
condition that had better solutions.

It is one of the reasons we have stuck with 1.9, and just merge  
decently designed improvements.

On Nov 3, 2007, at 7:35 PM, Chris Hostetter wrote:

> : This is simple not true. See FileDescriptor.sync().
> :
> : There are several options, but normally it is used so that when  
> close
> : completes, all data must be on disk. This is a much slower way to  
> write data.
> : It is very common in database systems when committing the log file.
> Ok.  I'll certainly take your word for it ... i've been trusting  
> the docs
> for [File]OutputStream.flush()...
>>> If the intended destination of this stream is an abstraction  
>>> provided
>>> by the underlying operating system, for example a file, then  
>>> flushing
>>> the stream guarantees only that bytes previously written to the  
>>> stream
>>> are passed to the operating system for writing; it does not  
>>> guarantee
>>> that they are actually written to a physical device such as a  
>>> disk drive.
> I haven't looked at the internals of FileOutputStream or  
> FileDescriptor on
> any particular platforms to see how exactly they work, but if  
> dealing with
> the FD directly and using FD.sync() the magic bullet then I'd love  
> to see
> a patch that uses it in FSDirectory.
> I assume the SyncFailedException it throws is rare?  If it is always
> thrown when using things like NFS that may be a show stopper for using
> sync() in Lucene ... many people have jumped through a lot of hoops  
> this
> past year to get Lucene working on NFS; I'd hate to see all that  
> work go
> out the window in an effort to make Lucene ACID.  (I suspect there are
> more users interested in using Lucene on NFS then on using it as a
> transactional data store)
>>> Throws:
>>>    SyncFailedException - Thrown when the buffers cannot be  
>>> flushed, or
>>> because the system cannot guarantee that all the buffers have been
>>> synchronized with physical media.
> -Hoss
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message