airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Massimo Menichinelli (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRFLOW-1355) Unable to launch jobs : DAGs not being executed.
Date Thu, 06 Jul 2017 10:45:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076336#comment-16076336
] 

Massimo Menichinelli edited comment on AIRFLOW-1355 at 7/6/17 10:44 AM:
------------------------------------------------------------------------

If I use Airflow (1.8.1) on my machine (macOS Sierra 10.12.5) by launching first the scheduler
and then the webserver with SequentialExecutor (i.e. basic default configuration), it works
fine. But with docker compose (https://github.com/puckel/docker-airflow), DAGs are launched
but not picked by scheduler / worker.
So not sure if the problem is with Airflow or its docker compose. It used to work well until
1-2 weeks ago (and the docker compose hasn't changed), not sure what has changed and caused
this.

I have:
Docker: Version 17.06.0-ce-mac18 (18433) Channel: stable
docker-compose: version 1.14.0, build c7bdf9e

I tried with previous versions of Docker: 1.12.6 (14937) and 17.03.1 (16048), the problem
is still the same.

DAGs are found and launched in the webserver, but scheduler and worker actually don't run
them and the process keeps running:

webserver_1  | [2017-07-06 09:42:20,577] [123] {models.py:168} INFO - Filling up the DagBag
from /usr/local/airflow/dags
worker_1     | [2017-07-06 09:42:21 +0000] [43] [INFO] Handling signal: ttou
worker_1     | [2017-07-06 09:42:21 +0000] [116] [INFO] Worker exiting (pid: 116)
webserver_1  | [2017-07-06 09:42:21 +0000] [42] [INFO] Handling signal: ttou
webserver_1  | [2017-07-06 09:42:21 +0000] [115] [INFO] Worker exiting (pid: 115)
scheduler_1  | [2017-07-06 09:42:22 +0000] [43] [INFO] Handling signal: ttin


was (Author: openp2pdesign):
If I use Airflow (1.8.1) on my machine (macOS Sierra 10.12.5) by launching first the scheduler
and then the webserver with SequentialExecutor (i.e. basic default configuration), it works
fine. But with docker compose (https://github.com/puckel/docker-airflow), DAGs are launched
but not picked by scheduler / worker.
So not sure if the problem is with Airflow or its docker compose. It used to work well until
1-2 weeks ago (and the docker compose hasn't changed), not sure what has changed and caused
this.

I have:
Docker: Version 17.06.0-ce-mac18 (18433) Channel: stable
docker-compose: version 1.14.0, build c7bdf9e

I tried with previous versions of Docker: 1.12.6 (14937) and 17.03.1 (16048), the problem
is still the same.

DAGs are found and launched in the webserver, but scheduler and worker actually don't run
them and the process keeps running:

{{webserver_1  | [2017-07-06 09:42:20,577] [123] {models.py:168} INFO - Filling up the DagBag
from /usr/local/airflow/dags
worker_1     | [2017-07-06 09:42:21 +0000] [43] [INFO] Handling signal: ttou
worker_1     | [2017-07-06 09:42:21 +0000] [116] [INFO] Worker exiting (pid: 116)
webserver_1  | [2017-07-06 09:42:21 +0000] [42] [INFO] Handling signal: ttou
webserver_1  | [2017-07-06 09:42:21 +0000] [115] [INFO] Worker exiting (pid: 115)
scheduler_1  | [2017-07-06 09:42:22 +0000] [43] [INFO] Handling signal: ttin}}

> Unable to launch jobs : DAGs not being executed.
> ------------------------------------------------
>
>                 Key: AIRFLOW-1355
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1355
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG
>    Affects Versions: Airflow 2.0
>         Environment: Mac OS and Ubuntu
>            Reporter: Pavan KN
>
> Steps to re-produce:
> 1. Create new installation
> 2. Launch Airflow
> 3. Enable a DAG and trigger it manually
> DAG/Job won't get executed. Will stay in Running status, but no execution starts and
continues to stay at same status.
> Same issues is there with Sequential, Local and Celeri executors.
> Happening in 2.0 version. Tried on multiple Mac machines and on Ubuntu.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message