lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomoko Uchida <tomoko.uchida.1...@gmail.com>
Subject Re: Question about usage of LuceneTestCase
Date Tue, 21 Aug 2018 14:47:18 GMT
Hi, Mike

Thanks for sharing your experiments.

> CommitsImplTest.testListCommits
> CommitsImplTest.testGetCommit_generation_notfound
> CommitsImplTest.testGetSegments
> DocumentsImplTest.testGetDocumentFIelds

I also found CommitsImplTest and DocumentsImplTest fail frequently,
especially CommitsImplTest is unhappy with lucene test framework (I pointed
that in my previous post.)

> I wonder if this is somehow related to running mvn from command line vs
running in IntelliJ since previously I was doing the latter

In my personal experience, when I was running those suspicious tests on
IntelliJ IDEA, they were always green - but I am not sure that `mvn test`
is the cause.

Thanks,
Tomoko

2018年8月21日(火) 22:53 Michael Sokolov <msokolov@gmail.com>:

> I was running these luke tests a bunch and found the following tests fail
> intermittently; pretty frequently. Once I @Ignore them I can get a
> consistent pass:
>
>
> CommitsImplTest.testListCommits
> CommitsImplTest.testGetCommit_generation_notfound
> CommitsImplTest.testGetSegments
> DocumentsImplTest.testGetDocumentFIelds
>
> I did not attempt to figure out why the tests were failing, but to do that,
> I would:
>
> Run repeatedly until you get a failure -- save the test "seed" from this
> run that should be printed out in the failure message Then you should be
> able to reliably reproduce this failure by re-running with system property
> "tests.seed" set to that value. This is used to initialize the
> randomization that LuceneTestCase does.
>
> My best guess is that the failures may have to do with randomly using some
> Directory implementation or other Lucene feature that Luke doesn't properly
> handle?
>
> Hmm I was trying this again to see if I could get an example, and strangely
> these tests are no longer failing for me after several runs, when
> previously they failed quite often. I wonder if this is somehow related to
> running mvn from command line vs running in IntelliJ since previously I was
> doing the latter
>
> -Mike
>
> On Tue, Aug 21, 2018 at 9:01 AM Tomoko Uchida <
> tomoko.uchida.1111@gmail.com>
> wrote:
>
> > Hello,
> >
> > Could you give me some advice or comments about usage of LuceneTestCase.
> >
> > Some of our unit tests extending LuceneTestCase fail by assertion error
> --
> > sometimes, randomly.
> > I suppose we use LuceneTestCase in inappropriate way, but cannot find out
> > how to fix it.
> >
> > Here is some information about failed tests.
> >
> >  * The full test code is here:
> >
> >
> https://github.com/DmitryKey/luke/blob/master/src/test/java/org/apache/lucene/luke/models/commits/CommitsImplTest.java
> >  * We run tests by `mvn test` on Mac PC or Travis CI (oracle jdk8/9/10,
> > openjdk 8/9/10), assertion errors occur regardless of platform or jdk
> > version.
> >  * Stack trace of an assertion error is at the end of this mail.
> >
> > Any advice are appreciated. Please tell me if more information is needed.
> >
> > Thanks,
> > Tomoko
> >
> >
> > -------------------------------------------------------
> >  T E S T S
> > -------------------------------------------------------
> > Running org.apache.lucene.luke.models.commits.CommitsImplTest
> > NOTE: reproduce with: ant test  -Dtestcase=CommitsImplTest
> > -Dtests.method=testGetSegmentAttributes -Dtests.seed=35AF58F652536895
> > -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=de
> > -Dtests.timezone=Africa/Kigali -Dtests.asserts=true
> > -Dtests.file.encoding=UTF-8
> > NOTE: leaving temporary files on disk at:
> >
> >
> /private/var/folders/xr/mrs6w1m15y1f4wkgfhn_x1dm0000gp/T/lucene.luke.models.commits.CommitsImplTest_35AF58F652536895-001
> > NOTE: test params are:
> >
> >
> codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION,
> > chunkSize=6, maxDocsPerChunk=7, blockSize=2),
> >
> >
> termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION,
> > chunkSize=6, blockSize=2)), sim=RandomSimilarity(queryNorm=true): {},
> > locale=de, timezone=Africa/Kigali
> > NOTE: Mac OS X 10.13.6 x86_64/Oracle Corporation 1.8.0_181
> > (64-bit)/cpus=4,threads=1,free=201929064,total=257425408
> > NOTE: All tests run in this JVM: [CommitsImplTest]
> > Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.44 sec
> > <<< FAILURE!
> >
> >
> testGetSegmentAttributes(org.apache.lucene.luke.models.commits.CommitsImplTest)
> > Time elapsed: 0.047 sec  <<< FAILURE!
> > java.lang.AssertionError
> > at
> __randomizedtesting.SeedInfo.seed([35AF58F652536895:AE37E8467BC01918]:0)
> > at org.junit.Assert.fail(Assert.java:92)
> > at org.junit.Assert.assertTrue(Assert.java:43)
> > at org.junit.Assert.assertTrue(Assert.java:54)
> > at
> >
> >
> org.apache.lucene.luke.models.commits.CommitsImplTest.testGetSegmentAttributes(CommitsImplTest.java:151)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> > at
> >
> >
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> > at
> >
> >
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> >
> >
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> >
> >
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> > at
> >
> >
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> > at
> >
> >
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > at
> >
> >
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> > at
> >
> >
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> > at
> >
> >
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> >
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> > at java.lang.Thread.run(Thread.java:748)
> >
>


-- 
Tomoko Uchida

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message