airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-679) Stop concurrent task instances from running due to race conditions
Date Fri, 16 Dec 2016 21:28:58 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15755551#comment-15755551
] 

ASF subversion and git services commented on AIRFLOW-679:
---------------------------------------------------------

Commit 51acc50875ee64af135caaf0583f51ef3868cda3 in incubator-airflow's branch refs/heads/master
from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=51acc50 ]

[AIRFLOW-679] Stop concurrent task instances from running

Check that PID remains unchanged, and throw
exception otherwise.

Testing Done:
- Ran a task, set PID to be different, and
ensuring it failed

If there's a connection error while heartbeating,
it should retry. Also,
if it hasn't been able to heartbeat for a while,
it should kill the
child processes so that we don't have 2 of the
same task running.

Closes #1939 from saguziel/consistency


> Stop concurrent task instances from running due to race conditions
> ------------------------------------------------------------------
>
>                 Key: AIRFLOW-679
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-679
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>            Priority: Minor
>
> Right now, multiple copies of the same task instance can run if someone clicks on the
UI multiple times. To fix this, I propose two things:
> 1) record hostname and pid in TaskInstance table, then when heartbeating, only continue
running if it matches



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message