airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-276) List of dags does not refresh in UI for a while
Date Mon, 08 Aug 2016 18:27:20 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412214#comment-15412214
] 

ASF subversion and git services commented on AIRFLOW-276:
---------------------------------------------------------

Commit 9d254a317dd54f555270ca568aff1cd0500e1e53 in incubator-airflow's branch refs/heads/master
from [~xuanji]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=9d254a3 ]

[AIRFLOW-276] Gunicorn rolling restart

- Tell gunicorn to prepend `[ready]` to worker process name once worker is ready (to serve
requests) - in particular this happens after DAGs folder is parsed
- Airflow cli runs gunicorn as a child process instead of `excecvp`-ing over itself
- Airflow cli monitors gunicorn worker processes and restarts them by sending TTIN/TTOU signals
to the gunicorn master process
- Fix bug where `conf.get('webserver', 'workers')` and `conf.get('webserver', 'webserver_worker_timeout')`
were ignored

- Alternatively, https://github.com/apache/incubator-airflow/pull/1684/files does the same
thing but the worker-restart script is provided separately for the user to run

- Start airflow, observe that workers are restarted
- Add new dags to dags folder and check that they show up
- Run `siege` against airflow while server is restarting and confirm that all requests succeed
- Run with configuration set to `batch_size = 0`, `batch_size = 1` and `batch_size = 4`

Closes #1685 from zodiac/xuanji_gunicorn_rolling_restart_2


> List of dags does not refresh in UI for a while
> -----------------------------------------------
>
>                 Key: AIRFLOW-276
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-276
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Li Xuanji
>            Assignee: Li Xuanji
>            Priority: Minor
>
> After creating a new dag (eg by adding a file to `~/airflow/dags`), the web UI does not
show the new for a while. It only shows it when either
> 1. gunicorn decides to restart the worker process, or
> 2. a scheduler picks up the new dag, adds it to the airflow db, and the web UI notices
it in the db



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message