lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: [JENKINS] Lucene-3.x - Build # 680 - Failure
Date Sat, 24 Mar 2012 12:11:09 GMT
I agree not being able to quickly reproduce a failure is bad, but,
worse, I don't want to lose test coverage.

The big line doc file has much more content (~200X larger) than the
checked in tiny one.  Some tests, especially when running nightly, can
fully consume the small line docs file and then they'll "wrap around"
and won't see unique terms anymore.

Does the failure reproduce if you use the big line docs file?
(-Dtest.linedocsfile=...)?  Ie, it's still reproducible, but it's a
hassle because you need the big line file docs locally?

If so... then maybe all that's missing is the "repro" line adding the
-Dtest.linedocsfile=/path/to/big/line/file/docs?  This way, when we
try to repro locally, we'd get a hard but clear error (because the
path presumably doesn't work in our local env), and we'd know to edit
that to point to our local copy?

Separately: does the test fail if you beast it using only the tiny
line docs file?

This really boils down to a test coverage vs test reproducibility
tradeoff... but I would favor coverage here: I'd rather know about a
bug, but struggle to reproduce/fix it, than not know about it at all.
Ignorance isn't bliss...

Mike McCandless

http://blog.mikemccandless.com

On Fri, Mar 23, 2012 at 11:36 PM, Robert Muir <rcmuir@gmail.com> wrote:
> This is likely a real bug (from added tests in LUCENE-3905)
>
> But, I cannot reproduce locally since I dont have the special linedocs file.
>
> I think we need to address this: we cant use random tests if they
> arent reproducible... hell i'd even bet happier if linedocsfile took a
> URL and the "reproduce-with" actually downloaded the huge file? But I
> think we can do better without this...
>
> For now maybe we should disable the special linedocs file in hudson?
> By using it, it means none of our analysis tests are reproducible
> anymore... which is a big loss.
>
> Index: hudson-lucene-trunk.sh
> ===================================================================
> --- hudson-lucene-trunk.sh      (revision 1304701)
> +++ hudson-lucene-trunk.sh      (working copy)
> @@ -33,6 +33,7 @@
>  #mv -f dist/*-src.tgz $ARTIFACTS
>
>  # run tests
> +# TODO: make reproducible  -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>  cd $WORKSPACE/$CORE_DIR
>  $ANT_HOME/bin/ant \
>   -Dsvnversion.exe=$SVNVERSION_EXE \
> @@ -40,12 +41,12 @@
>   -Dversion=$VERSION \
>   -Dtests.multiplier=$TESTS_MULTIPLIER \
>   -Dtests.nightly=true \
> -  -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>   -Dargs="$TEST_JVM_ARGS" \
>   test
>  # release it
>  mv -f dist/*.tgz $ARTIFACTS
>
> +# TODO: make reproducible -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>  cd $WORKSPACE/$MODULES_DIR
>  $ANT_HOME/bin/ant \
>   -Dsvnversion.exe=$SVNVERSION_EXE \
> @@ -53,7 +54,6 @@
>   -Dversion=$VERSION \
>   -Dtests.multiplier=$TESTS_MULTIPLIER \
>   -Dtests.nightly=true \
> -  -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>   -Dargs="$TEST_JVM_ARGS" \
>   test
>
> Index: hudson-lucene-3.x.sh
> ===================================================================
> --- hudson-lucene-3.x.sh        (revision 1304701)
> +++ hudson-lucene-3.x.sh        (working copy)
> @@ -24,13 +24,13 @@
>  mv -f build/docs/api/* $JAVADOCS_ARTIFACTS
>
>  # run tests
> +# TODO: make reproducible  -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>  JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
>   -Dsvnversion.exe=$SVNVERSION_EXE \
>   -Dsvn.exe=$SVN_EXE \
>   -Dversion=$VERSION \
>   -Dtests.multiplier=$TESTS_MULTIPLIER \
>   -Dtests.nightly=true \
> -  -Dtests.linedocsfile=$TEST_LINE_DOCS_FILE \
>   -Dargs="$TEST_JVM_ARGS" \
>   test
>  # release it
>
>
> On Fri, Mar 23, 2012 at 11:16 PM, Apache Jenkins Server
> <jenkins@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-3.x/680/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.analysis.charfilter.HTMLStripCharFilterTest.testRandomHugeStrings
>>
>> Error Message:
>> Uncaught exception by thread: Thread[Thread-10,5,]
>>
>> Stack Trace:
>> org.apache.lucene.util.UncaughtExceptionsRule$UncaughtExceptionsInBackgroundThread:
Uncaught exception by thread: Thread[Thread-10,5,]
>>        at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:83)
>>        at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:516)
>>        at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>>        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>>        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>>        at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146)
>>        at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
>>        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>>        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>>        at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
>>        at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36)
>>        at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
>>        at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>>        at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>        at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
>>        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
>>        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
>>        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:743)
>> Caused by: java.lang.AssertionError: finalOffset  expected:<992> but was:<991>
>>        at org.junit.Assert.fail(Assert.java:93)
>>        at org.junit.Assert.failNotEquals(Assert.java:647)
>>        at org.junit.Assert.assertEquals(Assert.java:128)
>>        at org.junit.Assert.assertEquals(Assert.java:472)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:182)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:563)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:50)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:335)
>>
>>
>>
>>
>> Build Log (for compile errors):
>> [...truncated 18091 lines...]
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message