lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3766) A test group and build plan for known unstable tests
Date Mon, 17 Sep 2012 21:23:08 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457344#comment-13457344
] 

Yonik Seeley commented on SOLR-3766:
------------------------------------

Some concerns:
- These tests should not be excluded from the normal non-jenkins command line "ant test"
- Some tests may only be unstable on our freebsd instance (because of undesirable configuration
like blackhole).  I would not want to see such tests excluded from failing the build on all
jenkins/automated tests.  This goes back to Mark's request for a dont-run-on-our-freebsd annotation
(which makes more sense now that we have other jenkins servers running).
- Some tests may only be unstable for certain JVMs

There should also be consensus before a test gets thrown into a place where people won't notice
(or care about) it's failure.  Without that, we'd need to come to consensus on an unambiguous
policy about when a test may be marked such that it will no longer fail builds.

Maybe an easier solution would still be to send mail on failure, and then people who were
not interested could exclude it in a mail filter.
                
> A test group and build plan for known unstable tests
> ----------------------------------------------------
>
>                 Key: SOLR-3766
>                 URL: https://issues.apache.org/jira/browse/SOLR-3766
>             Project: Solr
>          Issue Type: Task
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>            Priority: Minor
>
> As discussed on the mailing list and suggested by Hoss:
> {code}
>  1) slap a special "@UnstableTest" annotation on them
>  2) set up a new jenkins job that *only* runs these @UnstableTest jobs
>  3) configure this new jenkins job to not send any email
> {code}
> Point 1 is trivial, 2 I have filed an issue for in the runner --
> https://github.com/carrotsearch/randomizedtesting/issues/122
> that I will take care of shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message