cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Knighton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-10517) Make sure all unit tests run on CassCI
Date Tue, 20 Oct 2015 16:29:27 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965342#comment-14965342
] 

Joel Knighton edited comment on CASSANDRA-10517 at 10/20/15 4:29 PM:
---------------------------------------------------------------------

In summary, this works as designed.

We set a timeout in the ant junit test runner that works on the level of the forked JVM. A
JVM is forked for every test class, and the timeout is global rather than per-method.  Upon
timeout, the forked JVM errors out and no more tests from this class are run.

In order to run all the testcases in a test class, we would need to abandon the timeouts in
the forked JVM and switch to junit annotations.

There's a development philosophy debate attached to this.  On the one hand, running all the
tests in every circumstance makes it easier to identify the timed-out test cases more than
one at a time.  On the other hand, a failure is a failure, and a red light will show up for
the build in the case of a timed out.

For now, I think we leave this be and use it as additional impetus to fix our test problem.
In the event that we find timed out tests to be one of the last remaining issues because of
unpredictable runtime behavior in CassCI, we should consider moving to junit annotations.

EDIT: I'm going to leave this open a bit longer, since I expect there will be good discussion
on this topic, and I'd rather have it centralized.


was (Author: jkni):
In summary, this works as designed.

We set a timeout in the ant junit test runner that works on the level of the forked JVM. A
JVM is forked for every test class, and the timeout is global rather than per-method.  Upon
timeout, the forked JVM errors out and no more tests from this class are run.

In order to run all the testcases in a test class, we would need to abandon the timeouts in
the forked JVM and switch to junit annotations.

There's a development philosophy debate attached to this.  On the one hand, running all the
tests in every circumstance makes it easier to identify the timed-out test cases more than
one at a time.  On the other hand, a failure is a failure, and a red light will show up for
the build in the case of a timed out.

For now, I think we leave this be and use it as additional impetus to fix our test problem.
In the event that we find timed out tests to be one of the last remaining issues because of
unpredictable runtime behavior in CassCI, we should consider moving to junit annotations.

> Make sure all unit tests run on CassCI
> --------------------------------------
>
>                 Key: CASSANDRA-10517
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10517
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: Jim Witschey
>            Assignee: Joel Knighton
>              Labels: triage
>             Fix For: 3.0.0
>
>
> It seems that some Windows unit tests aren't run sometimes on CassCI, and there's no
error reporting for this. For instance, this test was introduced around the time build #38
would have happened, but has only run in builds #50-3 and #64:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_utest_win32/lastCompletedBuild/testReport/org.apache.cassandra.cql3/ViewTest/testPrimaryKeyIsNotNull/history/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message