lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
Date Thu, 02 May 2019 22:13:00 GMT


Michael McCandless commented on LUCENE-8785:

{quote}If there is another thread coming in after we locked the existent threadstates we just
issue a new one.
Yuck :(
{quote}I think we can just do what deleteAll() does today except of not dropping the schema
on the floor?
The thing is, I think erasing schema while under transaction is a useful feature of Lucene. 
I realize neither ES nor Solr expose deleteAll but I don't think that's a valid argument to
remove it from Lucene ;)
{quote}I want to understand the usecase for this. I can see how somebody wants to drop all
docs but basically droping all IW state on the floor is difficult in my eyes.
Well, imagine a user searching documents with diverse/varying fields, maybe arriving from
an external (not controlled by the developer) source.  And for some reason the index is reset
once per week, but the devs want to allow searching of the old index while the new index is
(slowly) built up.  But if something goes badly wrong, they need to be able to rollback (the
{{deleteAll}} and all subsequently added docs) to the last commit and try again later. 
If instead it succeeds, then a refresh/commit will switch to the new index atomically.

> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -----------------------------------------------------
>                 Key: LUCENE-8785
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 7.6
>         Environment: OpenJDK 1.8.0_202
>            Reporter: Michael McCandless
>            Priority: Minor
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64 cores), and
hit this random yet spooky failure:
> {noformat}
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterDelete
-Dtests.method=testDeleteAllNoDeadLock -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon\
> serts=true -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock <<<
>    [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError:
Captured an uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE, group=TGRP-TestIndexWriterDelete]
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
>    [junit4]    > Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException:
field number 0 is already mapped to field name "null", not "content"
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
>    [junit4]    >        at org.apache.lucene.index.TestIndexWriterDelete$
>    [junit4]    > Caused by: java.lang.IllegalArgumentException: field number 0
is already mapped to field name "null", not "content"
>    [junit4]    >        at org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(
>    [junit4]    >        at org.apache.lucene.index.FieldInfos$Builder.getOrAdd(
>    [junit4]    >        at org.apache.lucene.index.DefaultIndexingChain.getOrAddField(
>    [junit4]    >        at org.apache.lucene.index.DefaultIndexingChain.processField(
>    [junit4]    >        at org.apache.lucene.index.DefaultIndexingChain.processDocument(
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriter.updateDocuments(
>    [junit4]    >        at org.apache.lucene.index.IndexWriter.updateDocuments(
>    [junit4]    >        at org.apache.lucene.index.IndexWriter.addDocuments(
>    [junit4]    >        at org.apache.lucene.index.RandomIndexWriter.addDocument(
>    [junit4]    >        at org.apache.lucene.index.TestIndexWriterDelete${noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle thread safety
issue in this code ... this is a hairy part of Lucene ;)

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message