flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zentol <...@git.apache.org>
Subject [GitHub] flink pull request #4233: [FLINK-7047] [travis] Reorganize build profiles
Date Fri, 30 Jun 2017 12:53:35 GMT
GitHub user zentol opened a pull request:


    [FLINK-7047] [travis] Reorganize build profiles

    Builds on-top of #4226 and #4227.
    This PR reorganizes the test groups.
    1. core - executes tests for core, runtime, streaming-java, scala, etc
    2. libraries - executes tests for modules libraries and storm
    3. connectors - executes tests for connectors and wikiedits
    4. tests - executes tests for flink-tests and examples
    5. misc - executes tests for yarn, mesos, fs-tests, dist
    The main change (and worst part) is that every module (or their respective parent) now
have a `run-tests` profile that is activated when the specific property for that groups was
    For example, tests for `flink-libraries` are activated if the `flink.test.lib` property
is set.
    For this to work it was necessary to **disable the test execution by default**. Execution
of tests is now **strictly opt-in** for each module (or their parent).
    To keep the number of build profiles <= 10 I've also reduced the combinations of jdk+scala+hadoop
that we're testing to 2:
    * oraclejdk8 + scala11 + hadoop2.8.0
    * openjdk7 + scala10 + hadoop2.4.1
    For the jdk7 profile max build times hover around 45 minutes for the core tests; for jdk8
the same build takes 40 minutes.
    I tried to reduce this further my cutting out parts of the compilation, specifically the
scala modules which easily add 6-8 minutes, but i couldn't find a way that properly works
and is maintainable enough. Instead of fiddling with maven it's probably easier just to split
the entire repo and call it a day.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 7047

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4233
commit 94e2b5bf02c2f5e04ecfd7af18f063a773e603c3
Author: zentol <chesnay@apache.org>
Date:   2017-06-29T21:17:01Z

    [FLINK-7048] [travis] Define javadoc skipping in travis watchdog script

commit 1d1cc7549c262985f7d9b98f079297d40210c73d
Author: zentol <chesnay@apache.org>
Date:   2017-06-29T19:31:23Z

    [FLINK-7046] [travis] Hide download logging messages

commit e8cb22ff9bac40fe4df69206b390f9d7040ea150
Author: zentol <chesnay@apache.org>
Date:   2017-06-29T15:36:21Z

    [FLINK-7047] [travis] Reorganize build profiles


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message