lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3855) TestStressNRT failures (reproducible)
Date Thu, 08 Mar 2012 08:30:57 GMT


Dawid Weiss commented on LUCENE-3855:

Yep, there is something severely wrong in there, but I won't be able to figure it out on my
own. Don't get the logic in IndexWriter. But I tracked one of the above exceptions to this

    [junit] junit.framework.AssertionFailedError: info=_dm(4.0):cv6/4 isn't live
    [junit]     at org.apache.lucene.index.IndexWriter$ReaderPool.infoIsLive(
    [junit]     at org.apache.lucene.index.IndexWriter$ReaderPool.dropAll(
    [junit]     at org.apache.lucene.index.IndexWriter.closeInternal(
    [junit]     at org.apache.lucene.index.IndexWriter.close(
    [junit]     at org.apache.lucene.index.IndexWriter.close(
    [junit]     at org.apache.lucene.index.RandomIndexWriter.close(
    [junit]     at org.apache.lucene.index.TestStressNRT.test(

So, the case here is that infoIsLive attempts to check:

      int idx = segmentInfos.indexOf(info);
      assert idx != -1: "info=" + info + " isn't live";

I added tracing to segmentInfos when segments do get removed from the underlying array. Once
executed, I get the listing:

>>> Removing: _o1(4.0):Cv13/13
>>> Removing: _o0(4.0):Cv6/6
>>> Removing: _nu(4.0):C12/12
>>> Removing: _nw(4.0):Cv12/12
>>> Removing: _o9(4.0):c2/2
>>> Removing: _q2(4.0):C12/12
>>> Removing: _qc(4.0):Cv7/7
>>> Removing: _q6(4.0):C12/12
>>> Not found: _d9(4.0):Cv8/3

But that last segment is never on the list of removed segments. It was never added there in
the first place. The allocation stack for that segment is:

	at java.lang.Thread.getStackTrace(
	at org.apache.lucene.index.SegmentInfo.<init>(
	at org.apache.lucene.index.IndexWriter._mergeInit(
	at org.apache.lucene.index.IndexWriter.mergeInit(
	at org.apache.lucene.index.ConcurrentMergeScheduler.merge(
	at org.apache.lucene.index.IndexWriter.maybeMerge(
	at org.apache.lucene.index.IndexWriter.maybeMerge(
	at org.apache.lucene.index.IndexWriter.prepareCommit(
	at org.apache.lucene.index.IndexWriter.commitInternal(
	at org.apache.lucene.index.IndexWriter.commit(
	at org.apache.lucene.index.IndexWriter.commit(
	at org.apache.lucene.index.RandomIndexWriter.commit(
	at org.apache.lucene.index.TestStressNRT$

So this looks like a race condition between closing the writer and a concurrent merge scheduler?

Let me know if you need any further stacks/ tracing listings -- this is fairly easy to reproduce
on my machine and I can add anything.

> TestStressNRT failures (reproducible)
> -------------------------------------
>                 Key: LUCENE-3855
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Dawid Weiss
>            Priority: Minor
>             Fix For: 4.0
>         Attachments: output1.log, output2.log, output3.log, output4.log
> Build server logs. Reproduces on at least two machines.
> {noformat}
>     [junit] ------------- Standard Error -----------------
>     [junit] NOTE: reproduce with: ant test -Dtestcase=TestStressNRT -Dtestmethod=test
-Dtests.seed=69468941c1bbf693:19e66d58475da929:69e9d2f81769b6d0 -Dargs="-Dfile.encoding=UTF-8"
>     [junit] NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=true,coord=false):
{}, locale=ro, timezone=Etc/GMT+1
>     [junit] NOTE: all tests run in this JVM:
>     [junit] [TestStressNRT]
>     [junit] NOTE: Linux 3.0.0-16-generic amd64/Sun Microsystems Inc. 1.6.0_27 (64-bit)/cpus=2,threads=1,free=74960064,total=135987200
>     [junit] ------------- ---------------- ---------------
>     [junit] Testcase: test(org.apache.lucene.index.TestStressNRT):	Caused an ERROR
>     [junit] MockDirectoryWrapper: cannot close: there are still open files: {_ng.cfs=8}
>     [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
still open files: {_ng.cfs=8}
>     [junit] 	at
>     [junit] 	at org.apache.lucene.index.TestStressNRT.test(
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(
>     [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(
>     [junit] 	at org.apache.lucene.util.StoreClassNameRule$1.evaluate(
>     [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(
>     [junit] Caused by: java.lang.RuntimeException: unclosed IndexInput: _ng.cfs
>     [junit] 	at
>     [junit] 	at$1.openSlice(
>     [junit] 	at
>     [junit] 	at org.apache.lucene.codecs.lucene3x.TermInfosReader.<init>(
>     [junit] 	at org.apache.lucene.codecs.lucene3x.Lucene3xFields.<init>(
>     [junit] 	at org.apache.lucene.codecs.lucene3x.PreFlexRWPostingsFormat$1.<init>(
>     [junit] 	at org.apache.lucene.codecs.lucene3x.PreFlexRWPostingsFormat.fieldsProducer(
>     [junit] 	at org.apache.lucene.index.SegmentCoreReaders.<init>(
>     [junit] 	at org.apache.lucene.index.SegmentReader.<init>(
>     [junit] 	at org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getMergeReader(
>     [junit] 	at org.apache.lucene.index.IndexWriter.mergeMiddle(
>     [junit] 	at org.apache.lucene.index.IndexWriter.merge(
>     [junit] 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(
>     [junit] 	at org.apache.lucene.index.ConcurrentMergeScheduler$
>     [junit] 
>     [junit] 
>     [junit] Test org.apache.lucene.index.TestStressNRT FAILED
> {noformat}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message