cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rene Moser <m...@renemoser.net>
Subject Re: [DISCUSS] Move to Github
Date Sat, 19 Dec 2015 22:11:52 GMT


On 12/19/2015 07:57 PM, Remi Bergsma wrote:

> I disagree with testing based on complexity. You simply cannot know the implications
upfront, as that is why you run the tests. What seems small, can break it all.
> 
> Example:
> This commit seems an easy_fix, right? Just a findbugs issue resolved.
> https://github.com/apache/cloudstack/commit/6a4927f660f776bcbd12ae45f4e63ae2c2e96774

Ok, definition of easy is for _me_: fixing a typo in a debug log output,
adding license headers, fixing a comment. adding documentation.

So we just have minor and major, fine for me. But would you run
integration tests for adding missing license header?

> It was just merged indeed, exactly as you propose. But it did cause a major outage. And
I don’t even use HyperV.
> Details: https://github.com/apache/cloudstack/pull/761
> 
> This is why all PRs to 4.6 and 4.7 (200+) that we merged over the last couple of months
were tested against a real cloud. No easy fixes and minor changes. We always need to run the
full integration tests.

This would be the best case (and good job btw!) but for how long can you
made it without automation? We need automated tests, and if possible
full blown automated integration testing. I agree.

> When new functionality is proposed, there aren’t many people willing to write unit
and integration tests to cover it. Until that changes, testing can only guard whatever the
tests cover. And when we merge new stuff without tests, the total coverage goes down making
the tests less relevant. In fact, when we resolve a bug we should write a tests along with
it. I know of one guy that does that on a regular basis.

It is ~2016. No excuse. I know we can not test everything, we are
dealing with hardware. But I would rather say, "merge it, it covers
tests and they passed" then "merge it, it has 2 LGTM".

> It’s not so simple as it seems unfortunately.

It has never been simple, and I didn't say so, but it should be our goal
to get there. Right?

Regards
René

Mime
View raw message