Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Wed, 7 Dec 2011 23:08:40 +0000 (UTC)
From: "Ken McCracken (Commented) (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <116505916.51363.1323299320126.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1035521501.50586.1323289000153.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3627) CorruptIndexException on indexing
 after a failure occurs after segments file creation but before any bytes
 are written
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164816#comment-13164816 ] 

Ken McCracken commented on LUCENE-3627:
---------------------------------------

I have been reviewing https://issues.apache.org/jira/browse/LUCENE-3255 which seems related in that the error is encountered on the same section of SegmentInfos.java.
One way to fix this might be to change SegmentInfos.java as follows where int format = input.readInt(); to

        int format;
        try {
            format = input.readInt();
        } catch (IOException ioe) {
            if (input.length() == 0) {
                try {
                    input.close();
                } finally {
                    directory.deleteFile(segmentFileName);
                }
                return;
            }
            throw ioe;
        }

however, there are unit tests that seem to verify that no file deletions are happening at this low a level.  

                
> CorruptIndexException on indexing after a failure occurs after segments file creation but before any bytes are written
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3627
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3627
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 3.5
>         Environment: lucene-3.5.0, src download from GA release lucene.apache.org.
> Mac OS X 10.6.5, running tests in Eclipse Build id: 20100218-1602, 
> java version "1.6.0_24"
> Java(TM) SE Runtime Environment (build 1.6.0_24-b07-334-10M3326)
> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02-334, mixed mode)
>            Reporter: Ken McCracken
>            Priority: Critical
>         Attachments: TestCrashCausesCorruptIndex.java
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> FSDirectory.createOutput(..) uses a RandomAccessFile to do its work.  On my system the default FSDirectory.open(..) creates an NIOFSDirectory.  If createOutput is called on a segments_* file and a crash occurs between RandomAccessFile creation (file system shows a segments_* file exists but has zero bytes) but before any bytes are written to the file, subsequent IndexWriters cannot proceed.  The difficulty is that it does not know how to clear the empty segments_* file.  None of the file deletions will happen on such a segment file because the opening bytes cannot not be read to determine format and version.
> I will attempt to attach a Test file demonstrates the issue; place it in your 
> src/test/org/apache/lucene/store/
> directory and run the unit tests with JUnit4.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org