airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fokko Driesprong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-2922) Potential deal-lock bug in CeleryExecutor()
Date Wed, 22 Aug 2018 09:07:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fokko Driesprong resolved AIRFLOW-2922.
---------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

> Potential deal-lock bug in CeleryExecutor()
> -------------------------------------------
>
>                 Key: AIRFLOW-2922
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2922
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: executor
>            Reporter: Xiaodong DENG
>            Assignee: Xiaodong DENG
>            Priority: Major
>             Fix For: 2.0.0
>
>
> Code: [https://github.com/apache/incubator-airflow/blob/404be4b021e803239f48e4dd5705759cc77aeb92/airflow/executors/celery_executor.py] 
> Celery Task states normally change in ways like “PENDING -> STARTED -> SUCCESS/FAILURE”
([http://docs.celeryproject.org/en/latest/reference/celery.states.html]).
> In lines 107 and 108, it’s `task.state` rather than `state`, i.e. it will reflect the
latest real-time state of the Celery task.
> Let’s imagine: task state becomes “STARTED” (initial state is “PENDING”), then
the if-elif-else block will be triggered. It’s possible the Celery task state becomes “SUCCESS”
when whichever line between 94-105 is running. At line 108, the latest state of the task in
_*self.last_state*_ will be changed to “SUCCESS” rather than “STARTED” because it’s
referring to `task.state` rather than variable `state`.
> Then this task will be dead-locked because the if-elif-else block will never be triggered
for it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message