airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yajun Dong (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-934) airflow delayed the task to start
Date Thu, 02 Mar 2017 03:32:45 GMT
Yajun Dong created AIRFLOW-934:
----------------------------------

             Summary: airflow delayed the task to start 
                 Key: AIRFLOW-934
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-934
             Project: Apache Airflow
          Issue Type: Bug
          Components: DagRun
    Affects Versions: Airflow 1.6.2
            Reporter: Yajun Dong
            Priority: Blocker


we have a complex DAG which includes many tasks. but recently we found some tasks start delayed.
for instance: 

start_task(which will start at 00:00) --> create_cluster(will finished at 00:11) -->
wait_task(start at 00:16) , 

note: 
1. wait_task only has one upstream that is create_cluster 
2. the server that the airflow hosts has enough memory, and celeryd_concurrency is 20. 

below is the log of wait_task: 

[2017-03-02 00:16:39,602] {models.py:124} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:16:39,603] {models.py:197} INFO - Importing /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:16:39,613] {models.py:284} INFO - Loaded DAG <DAG: etl_prod>
[2017-03-02 00:16:40,333] {models.py:124} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:16:40,333] {models.py:197} INFO - Importing /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:16:40,345] {models.py:284} INFO - Loaded DAG <DAG: etl_prod>
[2017-03-02 00:16:40,373] {models.py:936} INFO - 
--------------------------------------------------------------------------------
New run starting @2017-03-02T00:16:40.369560
--------------------------------------------------------------------------------
[2017-03-02 00:16:40,402] {models.py:951} INFO - Queuing into pool None
[2017-03-02 00:22:31,161] {models.py:124} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:22:31,162] {models.py:197} INFO - Importing /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:22:31,172] {models.py:284} INFO - Loaded DAG <DAG: etl_prod>
[2017-03-02 00:22:31,863] {models.py:124} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:22:31,863] {models.py:197} INFO - Importing /home/ubuntu/airflow/dags/etl_prod/etl_prod.py
[2017-03-02 00:22:31,874] {models.py:284} INFO - Loaded DAG <DAG: etl_prod>
[2017-03-02 00:22:31,901] {models.py:936} INFO - 
--------------------------------------------------------------------------------
New run starting @2017-03-02T00:22:31.897547
--------------------------------------------------------------------------------
[2017-03-02 00:22:31,911] {models.py:974} INFO - Executing <Task(BashOperator): wait_mins>
on 2017-03-01 00:00:00
[2017-03-02 00:22:31,922] {bash_operator.py:52} INFO - tmp dir root location: 
/tmp



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message