airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuri Bendana (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-166) Unable to start webserver using systemd script
Date Tue, 24 May 2016 00:01:23 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yuri Bendana updated AIRFLOW-166:
---------------------------------
    Description: 
I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the webserver using
the systemd script.  This used to work fine.  The issue is that it starts and then just hangs,
no error is reported and it finally times out after about a minute.  I tried starting it from
the command line and it works fine without timing out.  I also ran it in daemon mode with
-D and again it seems to be fine.  Any thoughts on how to debug this?

Here's the log output:
{code}
May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:53 ybendana-linux airflow: ____________       _____________
May 23 16:27:53 ybendana-linux airflow: ____    |__( )_________  __/__  /________      __
May 23 16:27:53 ybendana-linux airflow: ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
May 23 16:27:53 ybendana-linux airflow: ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
May 23 16:27:53 ybendana-linux airflow: _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Starting
gunicorn 19.3.0
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Listening
at: http://0.0.0.0:8080 (15960)
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Using worker:
sync
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067] [INFO] Booting
worker with pid: 16067
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069] [INFO] Booting
worker with pid: 16069
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070] [INFO] Booting
worker with pid: 16070
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071] [INFO] Booting
worker with pid: 16071
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start operation timed out.
Terminating.
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070] [INFO] Worker
exiting (pid: 16070)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960] [INFO] Handling
signal: term
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071] [INFO] Worker
exiting (pid: 16071)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069] [INFO] Worker
exiting (pid: 16069)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067] [INFO] Worker
exiting (pid: 16067)
May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960] [INFO] Shutting
down: Master
May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver daemon.
May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service entered failed state.
May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.
{code}

  was:
I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the webserver using
the systemd script.  This used to work fine.  The issue is that it starts and then just hangs,
no error is reported and it finally times out after about a minute.  I tried starting it from
the command line and it works fine without timing out.  I also ran it in daemon mode with
-D and again it seems to be fine.  Any thoughts on how to debug this?

Here's the log output:

May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:53 ybendana-linux airflow: ____________       _____________
May 23 16:27:53 ybendana-linux airflow: ____    |__( )_________  __/__  /________      __
May 23 16:27:53 ybendana-linux airflow: ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
May 23 16:27:53 ybendana-linux airflow: ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
May 23 16:27:53 ybendana-linux airflow: _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Starting
gunicorn 19.3.0
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Listening
at: http://0.0.0.0:8080 (15960)
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Using worker:
sync
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067] [INFO] Booting
worker with pid: 16067
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069] [INFO] Booting
worker with pid: 16069
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070] [INFO] Booting
worker with pid: 16070
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071] [INFO] Booting
worker with pid: 16071
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997] {__init__.py:36} INFO -
Using executor LocalExecutor
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014] {models.py:154} INFO - Filling
up the DagBag from /opt/airflow/dags
May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start operation timed out.
Terminating.
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070] [INFO] Worker
exiting (pid: 16070)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960] [INFO] Handling
signal: term
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071] [INFO] Worker
exiting (pid: 16071)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069] [INFO] Worker
exiting (pid: 16069)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067] [INFO] Worker
exiting (pid: 16067)
May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960] [INFO] Shutting
down: Master
May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver daemon.
May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service entered failed state.
May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.



> Unable to start webserver using systemd script
> ----------------------------------------------
>
>                 Key: AIRFLOW-166
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-166
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: Airflow 1.7.1
>         Environment: CentOS 7
>            Reporter: Yuri Bendana
>
> I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the webserver using
the systemd script.  This used to work fine.  The issue is that it starts and then just hangs,
no error is reported and it finally times out after about a minute.  I tried starting it from
the command line and it works fine without timing out.  I also ran it in daemon mode with
-D and again it seems to be fine.  Any thoughts on how to debug this?
> Here's the log output:
> {code}
> May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
> May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444] {__init__.py:36} INFO
- Using executor LocalExecutor
> May 23 16:27:53 ybendana-linux airflow: ____________       _____________
> May 23 16:27:53 ybendana-linux airflow: ____    |__( )_________  __/__  /________   
  __
> May 23 16:27:53 ybendana-linux airflow: ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /|
/ /
> May 23 16:27:53 ybendana-linux airflow: ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/
|/ /
> May 23 16:27:53 ybendana-linux airflow: _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
> May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446] {models.py:154} INFO
- Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Starting
gunicorn 19.3.0
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Listening
at: http://0.0.0.0:8080 (15960)
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960] [INFO] Using
worker: sync
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067] [INFO] Booting
worker with pid: 16067
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069] [INFO] Booting
worker with pid: 16069
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070] [INFO] Booting
worker with pid: 16070
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071] [INFO] Booting
worker with pid: 16071
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876] {__init__.py:36} INFO
- Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950] {__init__.py:36} INFO
- Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972] {__init__.py:36} INFO
- Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997] {__init__.py:36} INFO
- Using executor LocalExecutor
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885] {models.py:154} INFO
- Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951] {models.py:154} INFO
- Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983] {models.py:154} INFO
- Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014] {models.py:154} INFO
- Filling up the DagBag from /opt/airflow/dags
> May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start operation timed
out. Terminating.
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070] [INFO] Worker
exiting (pid: 16070)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960] [INFO] Handling
signal: term
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071] [INFO] Worker
exiting (pid: 16071)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069] [INFO] Worker
exiting (pid: 16069)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067] [INFO] Worker
exiting (pid: 16067)
> May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960] [INFO] Shutting
down: Master
> May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver daemon.
> May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service entered failed
state.
> May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message