lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (LUCENE-3785) Replace ant macros for running concurrent tests with ant-junit4.
Date Wed, 15 Feb 2012 22:11:59 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208837#comment-13208837
] 

Dawid Weiss edited comment on LUCENE-3785 at 2/15/12 10:11 PM:
---------------------------------------------------------------

An output for full lucene tests under junit4. Note interesting corner cases:

1) assumption failed (status: IGNOR/A; ignored by assumption). It is reported twice -- once
by junit4, then on stderr by LuceneTestCase.

{noformat}
   [junit4] Running org.apache.lucene.index.TestIndexWriterReader
   [junit4] IGNOR/A 0.22s S1 | TestIndexWriterReader.testNoTermsIndex
   [junit4]    > Throwable #1: org.junit.internal.AssumptionViolatedException: got: <org.apache.lucene.util.InternalAssumptionViolatedException:
failed assumption: PreFlex codec does not support ReaderTermsIndexDivisor!>, expected:
null
   [junit4]   2> NOTE: Assume failed in 'testNoTermsIndex(org.apache.lucene.index.TestIndexWriterReader)'
(ignored): got: <org.apache.lucene.util.InternalAssumptionViolatedException: failed assumption:
PreFlex codec does not support ReaderTermsIndexDivisor!>, expected: null
   [junit4]   2>
   [junit4] Tests run:  20, Failures:   0, Errors:   0, Skipped:   1, Time: 17.25s
{noformat}

2) ignored tests, all reported under a suite:

{noformat}
   [junit4] Running org.apache.lucene.util.junitcompat.TestReproduceMessage
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeAfterClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureAfterClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorAfterClass
   [junit4] Tests run:  21, Failures:   0, Errors:   0, Skipped:   9, Time:  0.14s
{noformat}

3) I've added past statistics for tests for load balancing; they seem to do a fairly good
job:

{noformat}
   [junit4] Expected execution time on slave 0:    89.37s
   [junit4] Expected execution time on slave 1:    89.37s
   [junit4] Expected execution time on slave 2:    89.37s
   [junit4] Expected execution time on slave 3:    89.37s
{noformat}

and the result was:

{noformat}
   [junit4] Slave 0:     0.47 ..    81.83 =    81.36s
   [junit4] Slave 1:     0.47 ..    84.37 =    83.89s
   [junit4] Slave 2:     0.54 ..    81.92 =    81.38s
   [junit4] Slave 3:     0.47 ..    84.75 =    84.28s
   [junit4] Execution time total: 1 minute 24 seconds
   [junit4] Tests summary: 284 suites, 1587 tests, 25 ignored (12 assumptions)
{noformat}



                
      was (Author: dweiss):
    An output for full lucene tests under junit4. Note interesting corner cases:

1) assumption failed. It is reported twice -- once by junit4, then on stderr by LuceneTestCase.

{noformat}
   [junit4] Running org.apache.lucene.index.TestIndexWriterReader
   [junit4] IGNOR/A 0.22s S1 | TestIndexWriterReader.testNoTermsIndex
   [junit4]    > Throwable #1: org.junit.internal.AssumptionViolatedException: got: <org.apache.lucene.util.InternalAssumptionViolatedException:
failed assumption: PreFlex codec does not support ReaderTermsIndexDivisor!>, expected:
null
   [junit4]   2> NOTE: Assume failed in 'testNoTermsIndex(org.apache.lucene.index.TestIndexWriterReader)'
(ignored): got: <org.apache.lucene.util.InternalAssumptionViolatedException: failed assumption:
PreFlex codec does not support ReaderTermsIndexDivisor!>, expected: null
   [junit4]   2>
   [junit4] Tests run:  20, Failures:   0, Errors:   0, Skipped:   1, Time: 17.25s
{noformat}

2) ignored tests, all reported under a suite:

{noformat}
   [junit4] Running org.apache.lucene.util.junitcompat.TestReproduceMessage
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeAfterClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testAssumeBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testFailureAfterClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorBeforeClass
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorInitializer
   [junit4] IGNORED 0.00s S0 | TestReproduceMessage.testErrorAfterClass
   [junit4] Tests run:  21, Failures:   0, Errors:   0, Skipped:   9, Time:  0.14s
{noformat}

3) I've added past statistics for tests for load balancing; they seem to do a fairly good
job:

{noformat}
   [junit4] Expected execution time on slave 0:    89.37s
   [junit4] Expected execution time on slave 1:    89.37s
   [junit4] Expected execution time on slave 2:    89.37s
   [junit4] Expected execution time on slave 3:    89.37s
{noformat}

and the result was:

{noformat}
   [junit4] Slave 0:     0.47 ..    81.83 =    81.36s
   [junit4] Slave 1:     0.47 ..    84.37 =    83.89s
   [junit4] Slave 2:     0.54 ..    81.92 =    81.38s
   [junit4] Slave 3:     0.47 ..    84.75 =    84.28s
   [junit4] Execution time total: 1 minute 24 seconds
   [junit4] Tests summary: 284 suites, 1587 tests, 25 ignored (12 assumptions)
{noformat}



                  
> Replace ant macros for running concurrent tests with ant-junit4.
> ----------------------------------------------------------------
>
>                 Key: LUCENE-3785
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3785
>             Project: Lucene - Java
>          Issue Type: Sub-task
>          Components: general/build, general/test
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: failure-cases.patch, junit4-1.txt, junit4-2.txt, lucene-tests.txt,
trunk1.txt, trunk2.txt
>
>
> ant-junit4 is an ANT task for running tests in parallel (on slave JVMs). Its advantages
over the current macros:
> - dynamic load balancing based on historical test runs and current execution (job-stealing),
> - jvm-crash resilience (I wrote tests that actually crash a slave jvms to make sure such
an event is reported appropriately),
> - nicer console reporting (no need for syserrs on LuceneTestCase level).
> More documentation and info will follow as I roll out a patch.
> NOTE. The code for this issue is being developed at:
> https://github.com/dweiss/lucene_solr/tree/LUCENE-3785

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message