airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Meickle (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-2729) .airflowignore is not being respected
Date Mon, 09 Jul 2018 18:59:00 GMT
James Meickle created AIRFLOW-2729:
--------------------------------------

             Summary: .airflowignore is not being respected
                 Key: AIRFLOW-2729
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2729
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: 1.10
            Reporter: James Meickle


I have a repo that in 1.10 is giving airflowignore errors that did not exist in 1.9. I have
a DAG repo with the following .airflowignore:

{{airflow@XXX:~$ ls -la /home/airflow/airflow/dags/airflow-tasks/}}
{{total 172}}
{{drwxr-xr-x 6 airflow airflow 4096 Jul 9 18:48 .}}
{{drwxrwxr-x 3 airflow airflow 4096 Jul 9 18:48 ..}}
{{-rw-r--r-- 1 airflow airflow 13 Jul 9 16:20 .airflowignore}}

{{airflow@airflow-core-i-063df3268720e58fd:~$ cat /home/airflow/airflow/dags/airflow-tasks/.airflowignore
}}
{{submodules/*}}

However, the submoduled repository is being scanned for DAGs anyways, including the test suite.
Note the paths in the section below:

 

{{Jul 09 18:52:01 airflow_web-stdout.log: [2018-07-09 18:52:01,814] \{{models.py:351}} DEBUG
- Importing /home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/quantflow/operators/zipline_operators.py}}
{{Jul 09 18:52:01 airflow_web-stdout.log: [2018-07-09 18:52:01,817] \{{models.py:351}} DEBUG
- Importing /home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_sqs_operators.py}}
{{Jul 09 18:52:01 airflow_web-stdout.log: [2018-07-09 18:52:01,818] \{{models.py:365}} ERROR
- Failed to import: /home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_sqs_operators.py}}
{{Jul 09 18:52:01 airflow_web-stdout.log: Traceback (most recent call last):}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/airflow/models.py",
line 362, in process_file}}
{{Jul 09 18:52:01 airflow_web-stdout.log: m = imp.load_source(mod_name, filepath)}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/virtualenvs/airflow/lib/python3.5/imp.py",
line 172, in load_source}}
{{Jul 09 18:52:01 airflow_web-stdout.log: module = _load(spec)}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
693, in _load}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
673, in _load_unlocked}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap_external>",
line 665, in exec_module}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
222, in _call_with_frames_removed}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_sqs_operators.py",
line 6, in <module>}}
{{Jul 09 18:52:01 airflow_web-stdout.log: from moto import mock_sqs}}
{{Jul 09 18:52:01 airflow_web-stdout.log: ImportError: No module named 'moto'}}
{{Jul 09 18:52:01 airflow_web-stdout.log: [2018-07-09 18:52:01,821] \{{models.py:351}} DEBUG
- Importing /home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_zipline_operators.py}}
{{Jul 09 18:52:01 airflow_web-stdout.log: [2018-07-09 18:52:01,822] \{{models.py:365}} ERROR
- Failed to import: /home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_zipline_operators.py}}
{{Jul 09 18:52:01 airflow_web-stdout.log: Traceback (most recent call last):}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/airflow/models.py",
line 362, in process_file}}
{{Jul 09 18:52:01 airflow_web-stdout.log: m = imp.load_source(mod_name, filepath)}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/virtualenvs/airflow/lib/python3.5/imp.py",
line 172, in load_source}}
{{Jul 09 18:52:01 airflow_web-stdout.log: module = _load(spec)}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
693, in _load}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
673, in _load_unlocked}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap_external>",
line 665, in exec_module}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "<frozen importlib._bootstrap>", line
222, in _call_with_frames_removed}}
{{Jul 09 18:52:01 airflow_web-stdout.log: File "/home/airflow/airflow/dags/airflow-tasks/submodules/quantflow/tests/operators/test_zipline_operators.py",
line 6, in <module>}}
{{Jul 09 18:52:01 airflow_web-stdout.log: from freezegun import freeze_time}}
{{Jul 09 18:52:01 airflow_web-stdout.log: ImportError: No module named 'freezegun'}}

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message