lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4738) Killed JVM when first commit was running will generate a corrupted index
Date Thu, 11 Apr 2013 16:57:16 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629106#comment-13629106
] 

Michael McCandless commented on LUCENE-4738:
--------------------------------------------

bq. If I create a new IndexWriter with mode CREATE_OR_APPEND, and there is an empty segments_1
file, but other lucene files, will the IndexWriter throw an exception, or delete the never
committed lucene files?  What if there was an empty segments_5 file, would lucene delete the
files for the never committed segment_5?

You should get an exception from IW in both of these cases, unless you use OpenMode.CREATE.
                
> Killed JVM when first commit was running will generate a corrupted index
> ------------------------------------------------------------------------
>
>                 Key: LUCENE-4738
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4738
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0
>         Environment: OS: Linux 2.6.32-220.23.1.el6.x86_64
> Java: java version "1.7.0_05"
> Lucene: lucene-core-4.0.0 
>            Reporter: Billow Gao
>            Assignee: Michael McCandless
>             Fix For: 5.0, 4.3
>
>         Attachments: LUCENE-4738.patch, LUCENE-4738.patch, LUCENE-4738.patch, LUCENE-4738_test.patch
>
>
> 1. Start a NEW IndexWriterBuilder on an empty folder,
>    add some documents to the index
> 2. Call commit
> 3. When the segments_1 file with 0 byte was created, kill the JVM
> We will end with a corrupted index with an empty segments_1.
> We only have issue with the first commit crash.
> Also, if you tried to open an IndexSearcher on a new index. And the first commit on the
index was not finished yet. Then you will see exception like:
> ===========================================================================
> org.apache.lucene.index.IndexNotFoundException: no segments* file found in org.apache.lucene.store.MMapDirectory@C:\tmp\testdir
lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ee00df: files: [write.lock, _0.fdt,
_0.fdx]
> 	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:741)
> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:65)
> ===========================================================================
> So when a new index was created, we should first create an empty index. We should not
wait for the commit/close call to create the segment file.
> If we had an empty index there. It won't leave a corrupted index when there were a power
issue on the first commit. 
> And a concurrent IndexSearcher can access to the index(No match is better than exception).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message