airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-2368) Relative path should be used when enqueuing task instace
Date Mon, 23 Apr 2018 23:57:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xiao Zhu updated AIRFLOW-2368:
------------------------------
    Description: 
In our Airflow setup, scheduler and workers (connected via Celery) runs on different machines
and have different root dirs. Currently in airflow when scheduler enqueues a task instance
it uses the full absolute path of the DAG file, while the DAG is located at a different
location on our workers (because of different root dirs). As a result workers can't find the
DAG file.

When scheduler enqueues a task instances it should use its relative path (relative to DAGS_FOLDER).

  was:
In our Airflow setup, scheduler and workers (connected via Celery) runs on different machine
and have different root dir. Currently in airflow when scheduler enqueues a task instance
it uses the full absolute path of the DAG file, while the DAG is located at a different
location on our workers (because of different root dir). As a result workers can't find the
DAG file.

When scheduler enqueues a task instances it should use its relative path (relative to DAGS_FOLDER).


> Relative path should be used when enqueuing task instace
> --------------------------------------------------------
>
>                 Key: AIRFLOW-2368
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2368
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Xiao Zhu
>            Priority: Major
>
> In our Airflow setup, scheduler and workers (connected via Celery) runs on different
machines and have different root dirs. Currently in airflow when scheduler enqueues a task
instance it uses the full absolute path of the DAG file, while the DAG is located at a
different location on our workers (because of different root dirs). As a result workers can't
find the DAG file.
> When scheduler enqueues a task instances it should use its relative path (relative to
DAGS_FOLDER).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message