lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-1044) Behavior on hard power shutdown
Date Wed, 14 Nov 2007 10:25:43 GMT


Michael McCandless commented on LUCENE-1044:

Perhaps for the short-term, but long-term it would be better to find a solution that's both
reliable and doesn't have such a big performance impact.

Agreed.  I will default doSync back to false, for now.

We really don't need to sync until we commit. It would be interesting to know how much it
slows things to do that. As a quick hack we could try running the 'sync' command line program
at each commit.

I will test this as a hack first just to see how performance compares
to the current approach.

If performance looks good, then we might look into implementing this in pure Java, changing
FSDirectory.close() to queue FileDescriptors, add a background thread that syncs queued files,
and add a Directory.sync() method that blocks until the queue is empty.

Will do!

> Behavior on hard power shutdown
> -------------------------------
>                 Key: LUCENE-1044
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>         Environment: Windows Server 2003, Standard Edition, Sun Hotspot Java 1.5
>            Reporter: venkat rangan
>            Assignee: Michael McCandless
>             Fix For: 2.3
>         Attachments: LUCENE-1044.patch, LUCENE-1044.take2.patch, LUCENE-1044.take3.patch
> When indexing a large number of documents, upon a hard power failure  (e.g. pull the
power cord), the index seems to get corrupted. We start a Java application as an Windows Service,
and feed it documents. In some cases (after an index size of 1.7GB, with 30-40 index segment
.cfs files) , the following is observed.
> The 'segments' file contains only zeros. Its size is 265 bytes - all bytes are zeros.
> The 'deleted' file also contains only zeros. Its size is 85 bytes - all bytes are zeros.
> Before corruption, the segments file and deleted file appear to be correct. After this
corruption, the index is corrupted and lost.
> This is a problem observed in Lucene 1.4.3. We are not able to upgrade our customer deployments
to 1.9 or later version, but would be happy to back-port a patch, if the patch is small enough
and if this problem is already solved.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message