airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Song Liu <>
Subject Interesting things about how to know it's a DAG file
Date Thu, 10 May 2018 07:58:47 GMT

I just create a custom Dag class naming such as "MyPipeline" by extending the "DAG" class,
but Airflow is failed to identify this is a DAG file.

After digging into the Airflow implementation around the file:

# Heuristic that guesses whether a Python file contains an # Airflow DAG definition. might_contain_dag
= True if safe_mode and not zipfile.is_zipfile(file_path): with open(file_path, 'rb') as f:
content = might_contain_dag = all( [s in content for s in (b'DAG', b'airflow')])

So if the keyword "DAG" and "airflow" contained, it is a DAG file.

I don't know is there any other be more scientific way for this ?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message