airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <ja...@potiuk.com>
Subject Re: Using different python version in PythonVirtualEnv operator
Date Fri, 17 Sep 2021 14:08:16 GMT
Just create your own image by extending the image as explained here:
https://airflow.apache.org/docs/docker-stack/build.html.

Specifically this chapter
https://airflow.apache.org/docs/docker-stack/build.html#extending-the-image
is followed by some examples. You just need to install python 3.9 in the
way that you'd usually install it in debian.

J.

On Fri, Sep 17, 2021 at 12:35 PM Malte Brunst <
Brunst.Malte@schoenberger-group.com> wrote:

> Hi fellow Airflow users,
>
>
>
> in one of my DAGs I am executing a python function from a python module I
> developed via the PythonVirtualEnv operator.
>
> I want to be able to develop this module independently of Airflow, which
> means I also want to be able to use a different python version than the one
> from the Airflow version I am using.
>
> I am currently running the Docker Airflow image
> apache/airflow:2.1.1-python3.8
>
> When I try to install python 3.9 as a dependency into the PythonVirtualEnv
> operator it throws an error saying it cannot find the python version:
>
>
>
>
>
> How can I install an additional python version into my Airflow image that
> can be used in the PythonVirtualEnv operator but does not interfere with
> the python version that Airflow itself is using?
>
> Thank you very much for any help or input!
>
>
>
> Best
>
> Robert
>
>
>
>
>
>
>
>

Mime
View raw message