Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Subject: Re: Call for participation: Issue triaging and PR review/testing
To: dev <dev@cloudstack.apache.org>,
 "users@cloudstack.apache.org" <users@cloudstack.apache.org>
References: <CANLSSBU6UN-USZfQ7FPyAK5tJjEe5q8bFkVfREFA0rBUOtA07Q@mail.gmail.com>
 <3e09e6d6-05d6-b33a-b6ee-e4b64f9f665b@widodh.nl>
 <CA+FPaeZ9cQNj8YnbvBw1QcZPM2O4nbeiwyga1vBgcv8vrY7iSA@mail.gmail.com>
From: Rene Moser <mail@renemoser.net>
Message-ID: <efeded49-c9b8-9d3f-ccc6-f7060ddcfed9@renemoser.net>
Date: Wed, 13 Dec 2017 13:56:06 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <CA+FPaeZ9cQNj8YnbvBw1QcZPM2O4nbeiwyga1vBgcv8vrY7iSA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
archived-at: Wed, 13 Dec 2017 12:59:33 -0000

Hi all

On 12/13/2017 05:04 AM, Ivan Kudryavtsev wrote:
> Hello, devs, users, Rohit. Have a good day.
> 
> Rohit, you intend to freeze 4.11 on 8 january and, frankly speaking, I see
> risks here. A major risk is that 4.10 is too buggy and it seems nobody uses
> it actually right now in production because it's unusable, unfortunately,
> so we are planning to freeze 4.11 which stands on untested 4.10 with a lot
> of lacks still undiscovered and not reported. I believe it's a very
> dangerous way to release one more release with bad quality. Actually,
> marvin and units don't cover regressions I meet in 4.10. Ok, let's take a
> look at new one our engineers found today in 4.10:

So, the point is, how do we (users, devs, all) improve quality?

Marvin is great for smoke testing but CloudStack is dealing with many
infra vendor components, which are not covered by the tests. How can we
detect flows not covered by marvin?

For me, I decided (independent of this discussion) to write integration
tests in a way one would not expect, not following the "happy path":

Try to break CloudStack, to make a better CloudStack.

Put a chaos monkey in your test infra: Shut down storage, kill a host,
put latency on storage, disable network on hosts, make load on a host.
read only fs on a cluster wide primary fs. shut down a VR, remove a VR.

Things that can happen!

Not surprisingly I use Ansible. It has an extensive amount of modules
which can be used to battle prove anything of your infra. Ansible
playbooks are fairly easy to write, even when you are not used to write
code.

I will share my works when ready.

René