lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3627) CorruptIndexException on indexing after a failure occurs after segments file creation but before any bytes are written
Date Thu, 08 Dec 2011 11:48:39 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-3627:
---------------------------------------

    Attachment: LUCENE-3627.patch


Thanks Ken!

I attached a new patch, whittling back the test a bit (while still
showing the bug), fixing whitespace, etc.

I also removed the @author tags (we don't attach names to the source
code...).

I fixed the bug in two places.  First, when writing the segments file,
I moved the createOutput in SegmentInfos.java inside the try/finally
that deletes the file if an exception occurs.  This way we shouldn't
create 0-byte segments_N anymore.

Second, in IndexFileDeleter I ignore a given segments_N if it's 0
bytes; I don't delete the file at that point because IFD's normal
ref-counting will eventually remove it.

                
> CorruptIndexException on indexing after a failure occurs after segments file creation
but before any bytes are written
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3627
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3627
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 3.5
>         Environment: lucene-3.5.0, src download from GA release lucene.apache.org.
> Mac OS X 10.6.5, running tests in Eclipse Build id: 20100218-1602, 
> java version "1.6.0_24"
> Java(TM) SE Runtime Environment (build 1.6.0_24-b07-334-10M3326)
> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02-334, mixed mode)
>            Reporter: Ken McCracken
>            Assignee: Michael McCandless
>            Priority: Critical
>         Attachments: LUCENE-3627.patch, LUCENE-3627_initial_proposal.txt, TestCrashCausesCorruptIndex.java
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> FSDirectory.createOutput(..) uses a RandomAccessFile to do its work.  On my system the
default FSDirectory.open(..) creates an NIOFSDirectory.  If createOutput is called on a segments_*
file and a crash occurs between RandomAccessFile creation (file system shows a segments_*
file exists but has zero bytes) but before any bytes are written to the file, subsequent IndexWriters
cannot proceed.  The difficulty is that it does not know how to clear the empty segments_*
file.  None of the file deletions will happen on such a segment file because the opening bytes
cannot not be read to determine format and version.
> An initial proposed patch file is attached below.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message