airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <ja...@potiuk.com>
Subject PIP 21 officially supported for next version of Airflow
Date Tue, 27 Apr 2021 08:16:59 GMT
Hey everyone,

I just merged a change that likely addresses the last problems we had
with the latest version of PIP. So far the recommendation was (and
still is for Airflow 2.0.2) to use PIP 2.20.4 or below to install
Airflow. There were a few issues reported with 2.0.2 installing with
the newer version of PIP and I think I finally managed to solve the
last conflict. The new resolver in PIP is much better in preventing us
from having conflicting dependencies. In our Docker images PIP 21.1 is
already set as default.

The next version of Airflow should install with the latest available
PIP with any "extra" combination (we are still following the
--constraint mechanism).

Also the next version of Airflow should be easier for poetry/pipenv
users to install because the new resolver in PIP helps us to make sure
no new dependency conflicts creep-in. Though still the only official
way of reproducible installation with constraints is so far only
supported via PIP.

If there are any poetry/pipenv users who would like to contribute
description/instructions on how to use the constraints of Airflow and
use poetry/pipenv for "user" installation - I would love to accept
some PRs for that. Discussion about it here:
https://github.com/apache/airflow/pull/11950#issuecomment-827197437

Big thanks to Tzu-ping Chung from the PIP team for help and useful
discussions. We are one of the more complex projects with > 400
dependencies and I hope we will help to drive PIP 21 adoption :).

J,

-- 
+48 660 796 129

Mime
View raw message