airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yegor Andreenko (Jira)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-5466) Custom library setup.py fails airflow
Date Thu, 12 Sep 2019 13:27:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yegor Andreenko updated AIRFLOW-5466:
-------------------------------------
    Description: 
I have created custom library to manage dags description. It's common used functions like
get configuration, slack_hook definition, etc. 

For that library I created `setup.py` file inside my project(seems legit). This project is
supposed to be used like 

```

<project>/

                dags/

                        setup.py

                        dags_1.py

                        dags_N.py

                        config/.....

                        __init__.py 

                test/

```

And we usually deploy dags folder to airflow.

Airflow will crash `initdb`, `scheduler`, `webserver`, `list_dags` commands(all cli commands)
if it encounters `setup.py` file in `dags` folder with an error message:

```

usage: airflow [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
 or: airflow --help [cmd1 cmd2 ...]
 or: airflow --help-commands
 or: airflow cmd --help
 error: invalid command 'webserver'

```

Expected behaviour:

 - airflow shouldn't fail on `setup.py` file in `dags` folder

 - or have a more clear message what went wrong

  was:
I have created custom library to manage dags description. It's common used functions like
get configuration, slack_hook definition, etc. 

For that library I created `setup.py` file inside my project(seems legit). This project is
supposed to be used like 

```

<project>/

                dags/

                        setup.py

                        dags_1.py

                        dags_N.py

                        config/.....

                        __init__.py 

                test/

```

And we usually deploy dags folder to airflow.

Airflow will crash `initdb`, `scheduler`, `webserver` commands if it encounters `setup.py`
file in `dags` folder with an error message:

```

usage: airflow [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
 or: airflow --help [cmd1 cmd2 ...]
 or: airflow --help-commands
 or: airflow cmd --help
 error: invalid command 'webserver'

```

Expected behaviour:

 - airflow shouldn't fail on `setup.py` file in `dags` folder

 - or have a more clear message what went wrong


> Custom library setup.py fails airflow
> -------------------------------------
>
>                 Key: AIRFLOW-5466
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5466
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: cli
>    Affects Versions: 1.10.3, 1.10.4, 1.10.5
>            Reporter: Yegor Andreenko
>            Priority: Major
>
> I have created custom library to manage dags description. It's common used functions
like get configuration, slack_hook definition, etc. 
> For that library I created `setup.py` file inside my project(seems legit). This project
is supposed to be used like 
> ```
> <project>/
>                 dags/
>                         setup.py
>                         dags_1.py
>                         dags_N.py
>                         config/.....
>                         __init__.py 
>                 test/
> ```
> And we usually deploy dags folder to airflow.
> Airflow will crash `initdb`, `scheduler`, `webserver`, `list_dags` commands(all cli commands)
if it encounters `setup.py` file in `dags` folder with an error message:
> ```
> usage: airflow [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
>  or: airflow --help [cmd1 cmd2 ...]
>  or: airflow --help-commands
>  or: airflow cmd --help
>  error: invalid command 'webserver'
> ```
> Expected behaviour:
>  - airflow shouldn't fail on `setup.py` file in `dags` folder
>  - or have a more clear message what went wrong



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message