cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Shuler <mich...@pbandjelly.org>
Subject Should we make everything blue in Jenkins?
Date Fri, 14 Aug 2015 19:16:09 GMT
This is a prompt for Cassandra developers to discuss the alternatives 
and let Test Engineering know what you desire.

As discussed a few times in person, on irc, etc., there are a couple 
different ways we can run tests in Jenkins, particularly 
cassandra-dtest. The Cassandra developers are the committers to unit 
tests, so Test Engineering runs whatever is in the branch. If you'd like 
to make changes to unit tests to make things blue, just commit those!

Currently, we run dtests as 1), but we could do 2):

1) Run all dtests that don't catastrophically hang a server, pass or 
fail, and report the results.
2) Run only known passing dtests, skipping anything that fails - make it 
all blue on the main branch builds.

The biggest benefit is that dev branch builds should be easily 
recognizable as able to merge, if the dtest run is passing and blue. 
There is no comparison with the main branch build needing interpretation.

Test Eng has recently added the ability run *only* the skipped tests and 
has a a prototype job, trunk_dtest-skipped-with-require, to dig through. 
This could be set up for all main branch builds, moving anything that 
doesn't pass 100% to the -skipped job. This is perhaps the drawback with 
2) above: we're simply not going to run all the dtests on your dev 
branch. I don't think it makes sense to set up a -skipped dtest job on 
your dev branches. In addition, there's another job result set to go 
look at to properly evaluate the true state of a Cassandra branch or 
release. There may be other side effects - feel free to chime in.

I'm on a "disconnected" holiday until Monday Aug 24, so I won't have a 
chance to check in until then - the Test Eng team can field questions or 
clarifications, if needed.

-- 
Warm regards,
Michael

Mime
View raw message