lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-4738) Killed JVM when first commit was running will generate a corrupted index
Date Tue, 09 Apr 2013 22:00:17 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-4738:
---------------------------------------

    Attachment: LUCENE-4738.patch

New patch with several things:

  * I folded in Rob's patch on LUCENE-2727, to have MockDirWrapper
    sometimes throw IOExc in openInput and createOutput to get better
    test coverage of "out of file descriptors" like situations

  * Added a new TestIndexWriterOutOfFileDescriptors

  * Changes DirReader.indexExists back to before LUCENE-2812; I think
    it's just too dangerous to try to be too "smart" about whether an
    index exists or not, so now the method returns true if it sees any
    segments file.  (These "smarts" were causing failures in the new
    test, and caused LUCENE-4870).

  * Fixes IndexWriter so that if OpenMode is CREATE it will work even
    if a corrupt index is present.  But if it's CREATE_OR_APPEND, or
    APPEND then a corrupt index will cause an exc so app must manually
    resolve.

                
> Killed JVM when first commit was running will generate a corrupted index
> ------------------------------------------------------------------------
>
>                 Key: LUCENE-4738
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4738
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0
>         Environment: OS: Linux 2.6.32-220.23.1.el6.x86_64
> Java: java version "1.7.0_05"
> Lucene: lucene-core-4.0.0 
>            Reporter: Billow Gao
>         Attachments: LUCENE-4738.patch, LUCENE-4738.patch, LUCENE-4738_test.patch
>
>
> 1. Start a NEW IndexWriterBuilder on an empty folder,
>    add some documents to the index
> 2. Call commit
> 3. When the segments_1 file with 0 byte was created, kill the JVM
> We will end with a corrupted index with an empty segments_1.
> We only have issue with the first commit crash.
> Also, if you tried to open an IndexSearcher on a new index. And the first commit on the
index was not finished yet. Then you will see exception like:
> ===========================================================================
> org.apache.lucene.index.IndexNotFoundException: no segments* file found in org.apache.lucene.store.MMapDirectory@C:\tmp\testdir
lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ee00df: files: [write.lock, _0.fdt,
_0.fdx]
> 	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:741)
> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:65)
> ===========================================================================
> So when a new index was created, we should first create an empty index. We should not
wait for the commit/close call to create the segment file.
> If we had an empty index there. It won't leave a corrupted index when there were a power
issue on the first commit. 
> And a concurrent IndexSearcher can access to the index(No match is better than exception).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message