lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "selckin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3028) IW.getReader() returns inconsistent reader on RT Branch
Date Thu, 14 Apr 2011 19:30:05 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019996#comment-13019996
] 

selckin commented on LUCENE-3028:
---------------------------------

Seems to fail once every 6-8 runs quite consistently (at least i think this is the issue)

brachnes/realtime_search r1092329

{{
    [junit] Testsuite: org.apache.lucene.index.TestRollingUpdates
    [junit] Testcase: testUpdateSameDoc(org.apache.lucene.index.TestRollingUpdates):    Caused
an ERROR
    [junit] MockDirectoryWrapper: cannot close: there are still open files: {_ho.fdt=1, _ho.prx=1,
_ho.fdx=1, _ho.nrm=1, _j0.fdt=1, _ho.tis=1, _j0.fdx=1, _j0.tis=1, _j0.prx=1, _ho.frq=1, _ho.tvx=1,
_ho.tvd=1, _j0.nrm=1, _ho.tvf=1, _j0.frq=1, _j0.tvf=1, _j0.tvd=1, _j0.tvx=1}
    [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_ho.fdt=1, _ho.prx=1, _ho.fdx=1, _ho.nrm=1, _j0.fdt=1, _ho.tis=1, _j0.fdx=1,
_j0.tis=1, _j0.prx=1, _ho.frq=1, _ho.tvx=1, _ho.tvd=1, _j0.nrm=1, _ho.tvf=1, _j0.frq=1, _j0.tvf=1
, _j0.tvd=1, _j0.tvx=1}
    [junit]     at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:414)
    [junit]     at org.apache.lucene.index.TestRollingUpdates.testUpdateSameDoc(TestRollingUpdates.java:104)
    [junit]     at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1226)
    [junit]     at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1154)
    [junit] Caused by: java.lang.RuntimeException: unclosed IndexInput
    [junit]     at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:369)
    [junit]     at org.apache.lucene.store.Directory.openInput(Directory.java:122)
    [junit]     at org.apache.lucene.index.TermVectorsReader.<init>(TermVectorsReader.java:86)
    [junit]     at org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:236)
    [junit]     at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:495)
    [junit]     at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:629)
    [junit]     at org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:587)
    [junit]     at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:172)
    [junit]     at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:377)
    [junit]     at org.apache.lucene.index.DirectoryReader.doReopenFromWriter(DirectoryReader.java:419)
    [junit]     at org.apache.lucene.index.DirectoryReader.doReopen(DirectoryReader.java:432)
    [junit]     at org.apache.lucene.index.DirectoryReader.reopen(DirectoryReader.java:392)
    [junit]     at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:129)
    [junit] 
    [junit] 
    [junit] Testcase: testUpdateSameDoc(org.apache.lucene.index.TestRollingUpdates):    FAILED
    [junit] Some threads threw uncaught exceptions!
    [junit] junit.framework.AssertionFailedError: Some threads threw uncaught exceptions!
    [junit]     at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:521)
    [junit]     at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1226)
    [junit]     at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1154)
    [junit] 
    [junit] 
    [junit] Tests run: 2, Failures: 1, Errors: 1, Time elapsed: 6.649 sec
    [junit] 
    [junit] ------------- Standard Error -----------------
    [junit] NOTE: reproduce with: ant test -Dtestcase=TestRollingUpdates -Dtestmethod=testUpdateSameDoc
-Dtests.seed=-4094951767438954769:-1203905293622856057
    [junit] NOTE: reproduce with: ant test -Dtestcase=TestRollingUpdates -Dtestmethod=testUpdateSameDoc
-Dtests.seed=-4094951767438954769:-1203905293622856057
    [junit] The following exceptions were thrown by threads:
    [junit] *** Thread: Thread-103 ***
    [junit] java.lang.AssertionError: expected: org.apache.lucene.index.DocumentsWriterDeleteQueue@18635827but
was: org.apache.lucene.index.DocumentsWriterDeleteQueue@223074f3 false
    [junit]     at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:359)
    [junit]     at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:346)
    [junit]     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1367)
    [junit]     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1339)
    [junit]     at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:125)
    [junit] *** Thread: Thread-106 ***
    [junit] java.lang.AssertionError: expected: org.apache.lucene.index.DocumentsWriterDeleteQueue@18635827but
was: org.apache.lucene.index.DocumentsWriterDeleteQueue@223074f3 false
    [junit]     at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:359)
    [junit]     at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:346)
    [junit]     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1367)
    [junit]     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1339)
    [junit]     at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:125)
    [junit] NOTE: test params are: codec=PreFlex, locale=th, timezone=America/St_Kitts
    [junit] NOTE: all tests run in this JVM:
    [junit] [TestSearch, TestCharTermAttributeImpl, TestCheckIndex, TestConsistentFieldNumbers,
TestCrash, TestDeletionPolicy, TestDocumentWriter, TestIndexReaderCloneNorms, TestLongPostings,
TestPayloads, TestPerFieldCodecSupport, TestRollingUpdates]
    [junit] NOTE: Linux 2.6.37-gentoo amd64/Sun Microsystems Inc. 1.6.0_24 (64-bit)/cpus=8,threads=1,free=45768760,total=246743040
    [junit] ------------- ---------------- ---------------
    [junit] TEST org.apache.lucene.index.TestRollingUpdates FAILED
}}

> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
>                 Key: LUCENE-3028
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3028
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: Realtime Branch
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>             Fix For: Realtime Branch
>
>         Attachments: LUCENE-3028.patch, LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after
each update and asserted that is always sees only one document. Yet, this fails with current
branch since there is a problem in how we flush in the getReader() case. What happens here
is that we flush all threads and then release the lock (letting other flushes which came in
after we entered the flushAllThread context, continue) so that we could concurrently get a
new segment that transports global deletes without the corresponding add. They sneak in while
we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message