cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nitt10prashant <...@git.apache.org>
Subject [GitHub] cloudstack pull request: CLOUDSTACK-8975:skip test case if niciraN...
Date Fri, 23 Oct 2015 07:58:50 GMT
Github user nitt10prashant commented on the pull request:

    https://github.com/apache/cloudstack/pull/960#issuecomment-150501158
  
    @runseb
    "I personally fail to see the benefit of skipped tests versus a failed test"
    If i may ,
    skip  is a type of failure where you know in advance  about failure  , since you know
about the failure you just Handle them  in order to differentiate from a general failure(may
be product issue).So its all about categorizing  the failure  like expected failure and unexpected
failure 
    
    Its about how you see reports 
    If you accept skip as known failure then 100% success on BVT ~ 20% of skipped =80% success
with 20% of known failure ,at least not a product issue which you can say  without looking
at particular script .
    Without skip  you have to analyze each and every failure then you may come up with % of
product  issue and % of failure due to setup an all
    
    "While @nitt10prashant and @pvr9711 you seem to want 100% success on BVT even if it includes
some tests that are actually not run."
    My comment:
    1-Not run test cases are  the  one which are not picked up by runner may be coz of different
tags and those test cases will not appear  in your report 
    2-skipped test case are the one which are picked up by runner but due to non availability
of configurations or some other reasons  could not run completely  and they will appear in
your report .
    3-I want 100% success on BVT  without any skip  if all the setup config is available .incase
they are not i would prefer to skip them in order to find out real failure  which does not
include any failure because of setup/configuration  just by looking at my report .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message