cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasanna Santhanam <...@apache.org>
Subject [DISCUSS] (CLOUDSTACK-4971) Verifying Test Beds : prerequisites for launching tests
Date Mon, 28 Oct 2013 06:15:19 GMT
I like the concept of a health check before launching tests. But it's
a difficult distinction to make whether to run a specific test because
its required hardware component/external dependency is not
available/not ready in the deployment OR have the test ensure that the
deployment is suitable for its run.

So our current health check only ensures two things
- SystemVMs are up and running
- Default Templates are 'Ready' in the system

Take the example of LDAP here that you quoted. In such a case I'd
prefer that the test itself check for the deployment guarantees
without pushing that to the framework or the deployment scripts to
decipher.

There is a counter case - some tests use the netaddr module to do
subnet arithmetic. For these to run we need to ensure the virtualenv
under which the test module runs contains netaddr. This is something
that can be ensured by the deployment scripts.

So this is a question of guidelines to test authors on what to expect
and what not to expect when writing a test. Also how to have the
dependencies added automatically before the test gets executed. I
don't see a one-size-fits-all solution.

On Mon, Oct 28, 2013 at 05:49:39AM +0000, Santhosh Kumar Edukulla (JIRA) wrote:
> Santhosh Kumar Edukulla created CLOUDSTACK-4971:
> 
> 1, Currently test scripts are running despite the test setup has
> issues. EX: In one of the test scenario, where ldap server was down,
> then test suite depending upon them was still running. 
> 
> 2, Ideally, the setup should check for these dependencies before
> running the test script and exit wherever possible with proper log
> and reporting .
> 
> 3,  We may not be able to cover all setup issues, but the purpose of
> setup is also to check for its existence and dependencies. If
> scripts depending upon particular server conditions, should have a
> proper setup verifying the server existence before proceeding
> further. 
> 
> 4, I believe running test cases when setup is down is also not
> idealistic. 
> 
> 5. Purpose is to reduce script errors vs failures due to product
> bugs. 
> 
> 6. Logging this bug to track the setup issue cases or script
> failures. 
> 
-- 
Prasanna.,

------------------------
Powered by BigRock.com


Mime
View raw message