maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Weisgrab (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SUREFIRE-1556) Test XML file is not valid when rerun "fails" with an assumption
Date Tue, 21 Aug 2018 09:06:00 GMT

     [ https://issues.apache.org/jira/browse/SUREFIRE-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Weisgrab updated SUREFIRE-1556:
---------------------------------------
    Affects Version/s:     (was: 2.22.0)

> Test XML file is not valid when rerun "fails" with an assumption
> ----------------------------------------------------------------
>
>                 Key: SUREFIRE-1556
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1556
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>            Reporter: Michael Weisgrab
>            Priority: Blocker
>         Attachments: TEST-asdf.SearchTest_assumption_valid.xml, TEST-asdf.SearchTest_rerun_assumption_invalid.xml,
TEST-asdf.SearchTest_rerun_fail_valid.xml
>
>
> We are running UI tests and are doing one retry if there is a bug.
> When a test fails with an Exception on the first try and the second try is skipped by
an assumption violation the resulting test report xml file is not valid (not well formatted
xml file).
> When the test is skipped on the first try or both runs fail with an exception everything
is ok.
> This is a testcase which is retried and a valid xml is generated.
>  
> {code:java}
> <testcase name="xxx" classname="xxx" time="47.905">
>     <error message="message" type="org.openqa.selenium.TimeoutException">
>         ...
>     </error>
>     <system-out>
>         ...
>     </system-out>
>     <system-err>
>         ...
>     </system-err>
>     <rerunError message="message" type="java.lang.IndexOutOfBoundsException">
>         ...
>         <system-out>
>             ...
>         </system-out>
>         <system-err>
>             ...
>         </system-err>
>     </rerunError>
> </testcase>{code}
>  
>  
> Here is a generated invalid xml. The rerunError tag is missing. The retry "error" is
an assumption violation so the test should be marked as skipped with message of the assumptionviolation.
> {code:java}
> <testcase name="xxx" classname="xxx" time="54.153">
>   <error message="message" type="org.openqa.selenium.TimeoutException">
>   </error>
>   <system-out>
>     ...
>   </system-out>
>   <system-err>
>     ...
>   </system-err>
>   < message="There should be a list of ads!">
>     <system-out>
>       ...
>     </system-out>
>     <system-err>
>       ...
>     </system-err>
>   </>
> </testcase>
> {code}
> I have attached the following files:
>  * TEST-asdf.SearchTest_assumption_valid.xml (assumption violation in the first test
run)
>  * TEST-asdf.SearchTest_rerun_fail_valid.xml (both runs throw exception)
>  * TEST-asdf.SearchTest_rerun_assumption_invalid.xml (first run exception - second run
assumption violation)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message