lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@gmail.com>
Subject Re: Test failures are out of control......
Date Thu, 22 Feb 2018 06:55:15 GMT
Don't know, Yonik... If I make a change I am interested in regressions from
the start state; running flaky tests makes it impossible and frustrating
(and pointless in my opinion). I don't think my expectations are that much
off from the average - you may wake up being the only person who has the
defaults enabled, which seems wrong.

Dawid



On Feb 22, 2018 01:55, "Chris Hostetter" <hossman_lucene@fucit.org> wrote:

>
> : * Hoss has worked on aggregating all test failures from the 3 Jenkins
> : systems (ASF, Policeman, and Steve's), downloading the test results &
> logs,
> : and running some reports/stats on failures. He should be ready to share
> : this more publicly soon.
>
> I think Steve's linked to some of this before from jira comments, but it
> was only recently I realized i've never explicitly said to the list "Hey
> folks, here's a thing i've been working on" ...
>
>   http://fucit.org/solr-jenkins-reports/
>   https://github.com/hossman/jenkins-reports/
>
> The most interesting bit is probably here...
>
>   http://fucit.org/solr-jenkins-reports/failure-report.html
>
> ...but there are currently a few caveats:
>
> 1) there's some noise inthe '7days' data because I wasn't accounting for
> the way jenkins reports some types of failure -- that will gradually clean
> itself up
>
> 2) I think i've been been blocked by builds.apache.org, so at the moment
> the data seems to just be from the sarowe & policeman jenkins failures.
>
> 3) allthough the system is archiving the past 7 days worth of jenkins logs
> for any jobs with failures, there is currently no easy way to download
> the relevant log(s) from that failure report -- you currently have to
> download a CSV file like this one to corrolate the test failures to the
> jenkins job, and then go look for that job in the job-data dirs...
>
>   http://fucit.org/solr-jenkins-reports/reports/7days-method-failures.csv
>   http://fucit.org/solr-jenkins-reports/job-data/
>
> (My hope is to make #3 trivial from failure-report.html -- so you can say
> "hey weird, this test has failed X times, let's go download those logs."
> right from a single screen in your browser)
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message