lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-2576) Intermittent failure in TestIndexWriter.testCommitThreadSafety
Date Thu, 11 Nov 2010 11:25:13 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-2576:
---------------------------------------

    Attachment: LUCENE-2576.patch

OK I found the cause of the "expected:<1> but was:<0>" failures; it's
because SegmentInfos is failing to actually find the latest segments_N
file.

The test commits frequently and reopens readers frequently w/ multiple
threads.  Because our MockDirWrapper "acts like windows", we prevent
deleting segments_N files that are being read and so when a given
thread goes to reopen, the fallback logic in SegmentInfos can easily
open a too-old commit point.

To fix this I changed the retry order in which SegmentInfos tries to
find the segments file.  Previously, upon failing to load segments_N
we would immediately attempt to load segments_(N-1).  I've changed
that now to instead first try re-checking (re-list the dir and re-open
the segments.gen file), and only if re-checking produced the same "N"
do we then fallback to N-1.

I think this is actually an important change... back when this logic
was first created we did not have deletion policies and so a if the
segments_(N-1) existed it really did mean a commit to segments_N was
still in process.  But today, w/ del policy, it could easily mean that
segments_(N-1) was just not deleted.  So I think it's important to
re-list before trying segments_(N-1).

This fixes another failure case for this test, but, I still sometimes
see the "IndexFileDeleter does not know about XXX" (that other tests
have also hit), so I'll leave this open for now...


> Intermittent failure in TestIndexWriter.testCommitThreadSafety
> --------------------------------------------------------------
>
>                 Key: LUCENE-2576
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2576
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-2576.patch, LUCENE-2576.patch
>
>
> Mark's while(1) hudson box found this failure (and I can repro it too):
> {noformat}
> Error Message
> MockRAMDirectory: cannot close: there are still open files: {_1m.cfs=1,
> _1k.cfs=1, _14.cfs=1, _1g.cfs=1, _1h.cfs=1, _1f.cfs=1, _1n.cfs=1,
> _1i.cfs=1, _1j.cfs=1, _1l.cfs=1}
> Stacktrace
> java.lang.RuntimeException: MockRAMDirectory: cannot close: there are
> still open files: {_1m.cfs=1, _1k.cfs=1, _14.cfs=1, _1g.cfs=1,
> _1h.cfs=1, _1f.cfs=1, _1n.cfs=1, _1i.cfs=1, _1j.cfs=1, _1l.cfs=1}
>        at
> org.apache.lucene.store.MockRAMDirectory.close(MockRAMDirectory.java:282)
>        at
> org.apache.lucene.index.TestIndexWriter.testCommitThreadSafety(TestIndexWriter.java:4616)
>        at org.apache.lucene.util.LuceneTestCase.runBare(LuceneTestCase.java:328)
> Standard Output
> NOTE: random codec of testcase 'testCommitThreadSafety' was: Sep
> Standard Error
> The following exceptions were thrown by threads:
> *** Thread: Thread-1784 ***
> java.lang.RuntimeException: junit.framework.AssertionFailedError: null
>        at org.apache.lucene.index.TestIndexWriter$9.run(TestIndexWriter.java:4606)
> Caused by: junit.framework.AssertionFailedError: null
>        at junit.framework.Assert.fail(Assert.java:47)
>        at junit.framework.Assert.assertTrue(Assert.java:20)
>        at junit.framework.Assert.assertTrue(Assert.java:27)
>        at org.apache.lucene.index.TestIndexWriter$9.run(TestIndexWriter.java:4597)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message