lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3808) Switch LuceneTestCaseRunner to RandomizedRunner. Enforce Random sharing contracts. Enforce thread leaks.
Date Wed, 29 Feb 2012 09:17:58 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219016#comment-13219016
] 

Dawid Weiss commented on LUCENE-3808:
-------------------------------------

I've hacked a local branch where I substituted LuceneTestCaseRunner for RandomizedRunner.
And boy, hell broke loose ;)

So... the biggest issue I'm facing is indeed with Random sharing across threads. One may argue
that sharing a Random instance across threads (with data races) is in fact making it even
more random, but this doesn't seem to be the direction consistent with the possibility of
consistently re-running the tests given the same starting seed. I know it isn't possible to
coordinate threads anyway but I would still like to have Random instances to return identical
sequences from within a single thread's perspective.

After this lengthy introduction, here comes an example of Random sharing:
{noformat}
   > Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: This Random
was created for/by another thread (Thread[RandomizedRunner-SuiteThread-org.apache.lucene.search.TestSearcherManager-seed#[3F1A26A6DA253628],5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]).
Random instances must not be shared (acquire per-thread). Current thread: Thread[Thread-53,5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]
   > 	at __randomizedtesting.SeedInfo.seed([3F1A26A6DA253628]:0)
   > 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:307)
   > Caused by: java.lang.IllegalStateException: This Random was created for/by another
thread (Thread[RandomizedRunner-SuiteThread-org.apache.lucene.search.TestSearcherManager-seed#[3F1A26A6DA253628],5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]).
Random instances must not be shared (acquire per-thread). Current thread: Thread[Thread-53,5,RandomizedRunner-SuiteThreadGroup-org.apache.lucene.search.TestSearcherManager]
   > 	at com.carrotsearch.randomizedtesting.RandomNoSetSeed.checkValid(RandomNoSetSeed.java:124)
   > 	at com.carrotsearch.randomizedtesting.RandomNoSetSeed.nextInt(RandomNoSetSeed.java:72)
   > 	at org.apache.lucene.util.LuceneTestCase.rarely(LuceneTestCase.java:938)
   > 	at org.apache.lucene.analysis.MockAnalyzer.maybePayload(MockAnalyzer.java:102)
   > 	at org.apache.lucene.analysis.MockAnalyzer.createComponents(MockAnalyzer.java:95)
   > 	at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:83)
   > 	at org.apache.lucene.document.Field.tokenStream(Field.java:467)
   > 	at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:86)
   > 	at org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:287)
   > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
   > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
   > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1533)
   > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1505)
   > 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.updateDocument(ThreadedIndexingAndSearchingTestCase.java:111)
   > 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:259)
   > Caused by: com.carrotsearch.randomizedtesting.NotAnException: Original allocation
stack for this Random.
   > 	at java.lang.Thread.getStackTrace(Thread.java:1479)
   > 	at com.carrotsearch.randomizedtesting.RandomNoSetSeed.<init>(RandomNoSetSeed.java:26)
   > 	at com.carrotsearch.randomizedtesting.Randomness.<init>(Randomness.java:20)
   > 	at com.carrotsearch.randomizedtesting.Randomness.<init>(Randomness.java:24)
   > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.run(RandomizedRunner.java:531)
   > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:551)
   > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$3(RandomizedRunner.java:495)
   > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:472)
{noformat}

What's happening is that a Random is pushed to MockAnalyzer and then multiple threads access
the same instance. I am investigating possible workarounds for this -- Random instances are
in fact thread local in RandomizedRunner so it would be possible to just grab your own Random
in places like MockAnalyzer, without passing a ready to use instance. 

An ostrich solution is to dodge the issue and simply drop checking if a Random instance is
shared or not, but it wouldn't be so much fun then, would it?
                
> Switch LuceneTestCaseRunner to RandomizedRunner. Enforce Random sharing contracts. Enforce
thread leaks.
> --------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3808
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3808
>             Project: Lucene - Java
>          Issue Type: Sub-task
>          Components: general/test
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>            Priority: Minor
>             Fix For: 4.0
>
>
> Switch the runner to RandomizedRunner. Enforce the following:
> - Random sharing will result in a failure/ exception.
> - Thread leaks will result in a failure (add lingering if needed, but no ignores).
> - Add a validator for @Test method overrides (no-no).
> - Add a validator for static hook shadowing (no-no).
> - -Add a validator for testXXX without @Test annotation.-
> - Modify custom execution groups in LTC to be real @Groups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message