lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-8043) Attempting to add documents past limit can corrupt index
Date Thu, 30 Nov 2017 14:27:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yonik Seeley updated LUCENE-8043:
---------------------------------
    Attachment: YCS_IndexTest7a.java

The test code is just a modification of the previous code I was using.
I didn't think that test code would reproduce the issue for lucene-master, but I reverted
all my other changes to IW, and it does reproduce (w/o your patch)!  Uploaded YCS_IndexTest7a.java

This can often reproduce in as little as 4 documents indexed in 2 threads for me.
{code}
########## STARTING INDEXING RUN 0  IW.pendingNumDocs=0
########## IW.pendingNumDocs=2
ABOUT TO CALL commit
READER: reader.maxDoc=2 IW.pendingNumDocs=2
########## STARTING INDEXING RUN 1  IW.pendingNumDocs=2
########## IW.pendingNumDocs=0
ABOUT TO CALL commit
READER: reader.maxDoc=2 IW.pendingNumDocs=0
ERROR!!!!!!!!!!!!!!!!!!: reader.maxDoc=2 IW.pendingNumDocs=0
After sleep,commit,close reader.maxDoc=2 IW.pendingNumDocs=0
{code}

Still needs to be turned into a proper unit test, preferably w/o any sleeps.


> Attempting to add documents past limit can corrupt index
> --------------------------------------------------------
>
>                 Key: LUCENE-8043
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8043
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.10, 7.0, master (8.0)
>            Reporter: Yonik Seeley
>            Assignee: Simon Willnauer
>         Attachments: LUCENE-8043.patch, YCS_IndexTest7a.java
>
>
> The IndexWriter check for too many documents does not always work, resulting in going
over the limit.  Once this happens, Lucene refuses to open the index and throws a CorruptIndexException:
Too many documents.
> This appears to affect all versions of Lucene/Solr (the check was first implemented in
LUCENE-5843 in v4.9.1/4.10 and we've seen this manifest in 4.10) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message