airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject Request to test the new CI test environment / CI Docker Image
Date Sun, 07 Jul 2019 13:47:39 GMT
Hello everyone,

I think after the last round of reviews from Ash, I will be quite ready
this week to merge the final 3/3 step of the new CI Docker image.

Since it impacts integration-testing workflow of all Airflow developers, I
have a kind request to people who develop Airflow to test it before we
merge it to master.

The CI builds were extensively tested over the past few months so I am
quite confident it will be ok for Travis, but there are quite a few
improvements for local development.

Among others you will be able to:

   - Pull the CI images from DockerHub and build your local version that
   you will be use for testing via "local_ci_build.sh" (and force pull with
   another script)
   - Run "local_ci_run_airflow_testing.sh" <test_name> to run a test
   - Run "local_ci_enter_environment.sh" to enter the local IT environment
   being 1-1 to what you have in Travis CI and run tests more quickly (also
   with --skip-db-init flag for even faster runs)
   - Run static code analysis (mypy/pylint/flake8/doc build/licence check)
   in 1-1 the same way as you run them in Travis using "
   ci_pylint.sh/ci_mypy.sh ...."

I prepared the documentation update in CONTRIBUTING about "Integration test
development environment"
https://github.com/PolideaInternal/airflow/blob/ms-travis-ci-tests/CONTRIBUTING.md#integration-test-development-environment

You can check the branch out at
https://github.com/PolideaInternal/airflow/tree/ms-travis-ci-tests

There are more improvements to come/propose soon - pre-commits, possibly
full "simplified development environment" (which follows next as AIP-7),
production-optimised official Airflow image, but it is already quite a
change (improvement I believe) in the development workflow for Airflow.

I would love to hear if AIP-10 works as intended for you as well as for the
few people who used it/tested so far. Please let me know your
comments/questions - I am happy to answer them. Especially if you have
Linux desktop - I would love to get it tested.

If I have enough comments till Monday evening, I would love to merge it on
Tuesday.

J


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message