lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Updated: (LUCENE-2576) Intermittent failure in TestIndexWriter.testCommitThreadSafety
Date Thu, 11 Nov 2010 11:25:13 GMT


Michael McCandless updated LUCENE-2576:

    Attachment: LUCENE-2576.patch

OK I found the cause of the "expected:<1> but was:<0>" failures; it's
because SegmentInfos is failing to actually find the latest segments_N

The test commits frequently and reopens readers frequently w/ multiple
threads.  Because our MockDirWrapper "acts like windows", we prevent
deleting segments_N files that are being read and so when a given
thread goes to reopen, the fallback logic in SegmentInfos can easily
open a too-old commit point.

To fix this I changed the retry order in which SegmentInfos tries to
find the segments file.  Previously, upon failing to load segments_N
we would immediately attempt to load segments_(N-1).  I've changed
that now to instead first try re-checking (re-list the dir and re-open
the segments.gen file), and only if re-checking produced the same "N"
do we then fallback to N-1.

I think this is actually an important change... back when this logic
was first created we did not have deletion policies and so a if the
segments_(N-1) existed it really did mean a commit to segments_N was
still in process.  But today, w/ del policy, it could easily mean that
segments_(N-1) was just not deleted.  So I think it's important to
re-list before trying segments_(N-1).

This fixes another failure case for this test, but, I still sometimes
see the "IndexFileDeleter does not know about XXX" (that other tests
have also hit), so I'll leave this open for now...

> Intermittent failure in TestIndexWriter.testCommitThreadSafety
> --------------------------------------------------------------
>                 Key: LUCENE-2576
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>         Attachments: LUCENE-2576.patch, LUCENE-2576.patch
> Mark's while(1) hudson box found this failure (and I can repro it too):
> {noformat}
> Error Message
> MockRAMDirectory: cannot close: there are still open files: {_1m.cfs=1,
> _1k.cfs=1, _14.cfs=1, _1g.cfs=1, _1h.cfs=1, _1f.cfs=1, _1n.cfs=1,
> _1i.cfs=1, _1j.cfs=1, _1l.cfs=1}
> Stacktrace
> java.lang.RuntimeException: MockRAMDirectory: cannot close: there are
> still open files: {_1m.cfs=1, _1k.cfs=1, _14.cfs=1, _1g.cfs=1,
> _1h.cfs=1, _1f.cfs=1, _1n.cfs=1, _1i.cfs=1, _1j.cfs=1, _1l.cfs=1}
>        at
>        at
> org.apache.lucene.index.TestIndexWriter.testCommitThreadSafety(
>        at org.apache.lucene.util.LuceneTestCase.runBare(
> Standard Output
> NOTE: random codec of testcase 'testCommitThreadSafety' was: Sep
> Standard Error
> The following exceptions were thrown by threads:
> *** Thread: Thread-1784 ***
> java.lang.RuntimeException: junit.framework.AssertionFailedError: null
>        at org.apache.lucene.index.TestIndexWriter$
> Caused by: junit.framework.AssertionFailedError: null
>        at
>        at junit.framework.Assert.assertTrue(
>        at junit.framework.Assert.assertTrue(
>        at org.apache.lucene.index.TestIndexWriter$
> {noformat}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message