kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manasvi Gupta (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (KAFKA-4594) Annotate integration tests and provide gradle build targets to run subsets of tests
Date Thu, 16 Mar 2017 12:48:41 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manasvi Gupta reassigned KAFKA-4594:
------------------------------------

    Assignee: Manasvi Gupta

> Annotate integration tests and provide gradle build targets to run subsets of tests
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-4594
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4594
>             Project: Kafka
>          Issue Type: Bug
>          Components: unit tests
>            Reporter: Ewen Cheslack-Postava
>            Assignee: Manasvi Gupta
>            Priority: Minor
>              Labels: newbie
>
> It's possible to annotate unit tests and control which ones run based on these annotations.
Our tests have grown to take quite a long time because we've added a lot of relatively expensive
integration tests. Because these tests are all interleaved with the true unit tests and block
things like checkstyle from running, we've ended up with pretty bad turn around time on feedback
on simple PR issues.
> We could make this a lot nicer, and encourage users to actually run some tests before
submitting PRs with problems, by making it easier to get lightweight feedback based only on
unit tests. We'd want to annotate integration tests (which can probably mostly be accomplished
by annotating just a few shared {{*TestHarness}} classes and then setting up the build tasks
and dependencies to run things in fast-fail, more user-friendly order. In particular, it'd
be nice to:
> a) Have gradle tasks that can run just the unit tests + checkstyle. This provides devs
a *much* faster way to get at least basic feedback about whether there are problems with their
PR. PR Jenkins builds will also give much faster feedback (and avoid clogging up the Jenkins
slaves with lots of unnecessary work that will ultimately just fail on a unit test anyway).
> b) Have the order of tests be 1) build everything and run their unit tests, 2) run checkstyle
against all components, then 3) run integration tests. This could help address one of the
frustrating things about checkstyle tests today, which are put after all tests to avoid annoying
failures during development, but also results in a lot of folks missing simple checkstyle
issues before submitting PRs.
> The two obvious options I see for this are:
> 1. In JUnit4 (which we currently use), use Categories. These seem to unfortunately still
be under an experimental package. We pin to a specific version of JUnit, so this may not be
a huge issue. But it does mean any future migration would be more painful as the annotations
& integration with the test runner would need to change in the future.
> 2. Wait for JUnit5 with its built-in Tag support. This would be tied to making JDK8 the
minimum supported Java version. Given current plans that would mean we couldn't do this until
at least 0.11.0.0.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message