lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <>
Subject Re: [jira] Resolved: (LUCENE-1044) Behavior on hard power shutdown
Date Sun, 04 Nov 2007 15:49:48 GMT

"Yonik Seeley" <> wrote:

> On 11/4/07, Michael McCandless <> wrote:
> > The problem is, on a hard shutdown (kill -9 or JVM/machine crashes),
> > apparently future operations may have completed while some past
> > operations have not.  For example, the new segments_N file was
> > successfully written while say the _X.fdx file of the just-flushed
> > segment was not successfully written, even though Lucene had written &
> > closed _X.fdx before segments_N.
> That should be impossible except for a machine crash.  Kill -9 or a
> JVM crash should have no effect on data already written.

OK, right.  JVM crashing or getting killed should preserve
order-of-completion on the IO operations: those IO operations that
were handed off to the OS will eventually complete successfully.

But OS crashing, machine crashing or power-cord gets pulled can result
in out-of-order completion of IO operations, which is what can corrupt
the index.

> But a sync option would be both simple and useful for people trying to
> take live snapshots of an index, or to protect against machine
> crashes.  This isn't an absolute 100% guarantee either (so don't test
> for it) - the drives often lie to the OS about data being flushed.
> It's the best we can do at our level though.

Right, the best the OS can do is get all writes out to the drives, but
if the drives then cache the writes (in non-stable storage) then we
are still at risk.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message