airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Driesprong, Fokko" <fo...@driesprong.frl>
Subject Re: Big change in the way we run tests (AIP-10 completed)
Date Thu, 18 Jul 2019 12:15:33 GMT
Great work Jarek! Finally we've removed Tox!

Op wo 17 jul. 2019 om 22:54 schreef Jarek Potiuk <Jarek.Potiuk@polidea.com>:

> Hello everyone,
>
> I would like to announce that I just merged the last (3/3) part of AIP-10 -
> big change in the way we run CI tests for Airflow.
>
> Nothing changes in the tests, but the way we run them in Travis CI and
> locally is quite different. You should be able to reproduce Travis CI tests
> super-easily on your local Linux/MacOS development machine now following
> the documentation
> <
> https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#integration-test-development-environment
> >
> .
> The change gets rid of tox, Dockerises everything including
> pylint/mypy/flake checks, and optimises it so that it can be conveniently
> run both in CI and locally. Including faster iterations for re-running
> tests several times.
>
> Travis CI builds are now better and show python version + they print
> summary of failed tests at the end of the log. We also have a way to run
> static code check super-easily locally see .
>
> I truly hope it will make it easier for casual contributors to contribute
> to Airflow but also a lot of experienced Airflowers might find it really
> helpful. Please rebase to master and make a good use of it.
>
> The CONTRIBUTING.md documentation
> <
> https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#integration-test-development-environment
> >
> has changed a lot (and hopefully vastly improved).I expect some (hopefully
> minor) teething problems - in case you have any troubles please follow the
> troubleshooting
> <
> https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#troubleshooting
> >
> section of the documentation - if you cannot solve it following it, reach
> out to me on Slack or here in the discussion. I will try to be available as
> much as I can over the next few days to help in case of problems.
>
> There are few follow-up PRs/Proposal that are in the making for which
> AIP-10 was prerequisite (Breeze - simplified development environment,
> Pre-commit hooks) - also we will try to migrate out of Travis CI because of
> the recent problems with it (AIP-10 was instrumental to enable fast
> migration out of Travis) .
>
> IMPORTANT! Together with Ash we are going to run webinar/screencast next
> week (*Thursday 25th of July, 6.30pm CEST, 5.30pm London, 12:30 pm NYC,
> 9.30am California*). It will present what's already merged in AIP -10 and
> some of the improvements coming (Breeze/pre-commit). There will be separate
> announcement tomorrow about it so stay tuned.
>
> Big thanks to everyone who helped with this (especially to Ash for thorough
> reviews and great comments, but also to Fokko, Kamil, Gerardo for their
> support and comments - especially those critical ones that helped to make
> it better. Also Daniel, Stefan for testing during last two weeks. It's
> great community we have here :).
>
> BTW. I think this was the longest PR I ever worked on. I wrote the first
> version of the proposal
> <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-10+Multi-layered+and+multi-stage+official+Airflow+CI+image
> >
> mid-January
> and I worked on it since then.
> Fun fact -  I hold now the 2nd place for PR with the biggest number of
> comments (278)  in Airflow right after "[AIRFLOW-3585] - Add edges to
> database" (284).
>
> J.
>
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message