Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of simon.willnauer@gmail.com
 designates 209.85.219.48 as permitted sender)
MIME-Version: 1.0
Reply-To: simon.willnauer@gmail.com
Date: Fri, 9 Nov 2012 11:36:30 +0100
Message-ID: 
 <CAAHmpkgkhET8oryj7z_sp0xrKRUZmq3qRVdsMxp2N=E8MTwQOA@mail.gmail.com>
Subject: Failing tests aka. "what's the point of running them?"
From: Simon Willnauer <simon.willnauer@gmail.com>
To: dev@lucene.apache.org
Content-Type: text/plain; charset=UTF-8

hey folks,

I know yonik and mark had a long time power outage so I don't want to
blame anybody here but we need to fix those test failures.
Really when you look at those jenkins jobs:

http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/
https://builds.apache.org/computer/lucene/
https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/

its really funny if that'd be a joke but it isn't. If I'd be a new
contributor I'd be scared as sh** when I subscribe to the mailing
list. It's also not a good advertisement for us either. Yet, even
further it makes me miss failures I might have caused since the amount
of failure mails don't encourage to look at them since its the same
tests that fail over and over again no matter what code I commit. I
can already hear somebody saying "why don't you fix it" - well fair
enough but this project is massive and we are a large committer base
and I don't see myself fix the code I have never ever touched. Anyhow,
I really ask myself what is the point of running these tests,
specifically the solr ones, if the fail over and over again and nobody
cares? Even if folks care they don't get fixed and this project as
more committers than yonik and mark. Its really a bad sign if we are
at the point where we rely on 2 people to fix tests on a stable
branch...
I'd really like to hear how people want to address this, I mean it
would be just fair to disable the tests until somebody has the
patience / time to fix them we can / should make it blockers for a
release. Really a jenkins mail should be the exception not the
rule...I also think if the FreeBSD jenkins black hole stuff is a
problem for the tests then lets add a @BlackHoleProne annotation and
only run that on linux? I really don't care how we fix it but if we
don't have a solution by the end of next week I will add @Ignore to
all of them that failed in the last 2 weeks.

Sorry I got so frustrated about this - this is really bad press here!

simon

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org