airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Arnold (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-2338) Airflow requires Hive config
Date Wed, 18 Apr 2018 19:12:00 GMT
John Arnold created AIRFLOW-2338:
------------------------------------

             Summary: Airflow requires Hive config
                 Key: AIRFLOW-2338
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2338
             Project: Apache Airflow
          Issue Type: Bug
          Components: configuration
    Affects Versions: 1.9.0
            Reporter: John Arnold


Had working airflow, upgraded to latest from master branch, started getting config exceptions:

[2018-04-16 23:52:11,732] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 [2018-04-16
23:52:11,731] \{__init__.py:50} INFO - Using executor CeleryExecutor
[2018-04-16 23:52:12,478] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 Traceback
(most recent call last):
[2018-04-16 23:52:12,478] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/bin/airflow", line 32, in <module>
[2018-04-16 23:52:12,478] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 args.func(args)
[2018-04-16 23:52:12,478] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 77, in wrapper
[2018-04-16 23:52:12,478] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 raise
e
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 return
f(*args, **kwargs)
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 438, in run
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 conf.set(section,
option, value)
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/lib/python3.6/site-packages/backports/configparser/__init__.py", line
1239, in set
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 super(ConfigParser,
self).set(section, option, value)
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File
"/var/lib/celery/venv/lib/python3.6/site-packages/backports/configparser/__init__.py", line
921, in set
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 raise
from_none(NoSectionError(section))
[2018-04-16 23:52:12,479] \{base_task_runner.py:106} INFO - Job 147: Subtask login_task0 backports.configparser.NoSectionError:
No section: 'hive'
[2018-04-16 23:52:36,727] \{logging_mixin.py:95} INFO - [2018-04-16 23:52:36,726] \{jobs.py:2548}
INFO - Task exited with return code 1

It's looking for a hive section in the airflow.cfg but my config doesn't have one.

Not sure what changed to make this break, but the default values should be used / we should
not error out on a missing section that isn't even needed or used by current dags.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message