airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Vergiliev (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1108) DAG.fileloc is wrong when DAG is created from a factory method
Date Wed, 12 Apr 2017 14:30:41 GMT
Ivan Vergiliev created AIRFLOW-1108:
---------------------------------------

             Summary: DAG.fileloc is wrong when DAG is created from a factory method
                 Key: AIRFLOW-1108
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1108
             Project: Apache Airflow
          Issue Type: Bug
          Components: DAG, models
    Affects Versions: Airflow 1.8
            Reporter: Ivan Vergiliev
         Attachments: dag_factory_fileloc.patch

When using a factory method that creates DAGs, the created DAG's `fileloc` is set to the file
that contains the factory method instead of the file that declares the DAG. For example:

# dag_factory.py
    def create_dag(dag_id):
       dag = DAG(dag_id=dag_id, [...])
       [...]
       return dag

# dag_from_factory.py
    dag1 = create_dag('dag1')

In this case, dag1.fileloc will be `dag_factory.py` instead of the correct `dag_from_factory.py`.
This causes the Code tab to display the factory code instead of the definition code. More
importantly though, it also causes problems when refreshing the DAG in `DagBag.get_dag` -
it tries to reload the file that contains the factory method, however this file doesn't contain
any DAG definitions itself. This causes the webserver to think that the DAG does not actually
exist.

AFAICT this is a regression from this fix for SubDAGs: https://issues.apache.org/jira/browse/AIRFLOW-365
. I'm attaching a patch that adds a test for this. (Not creating it as a PR as it will obviously
cause the unit tests to fail.) If I have time, I might try fixing it as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message