airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-1166) _change_state_for_tis_without_dagrun can take very long and end up in deadlocks
Date Wed, 07 Jun 2017 07:17:18 GMT


ASF subversion and git services commented on AIRFLOW-1166:

Commit 4764646b18f56c34a35c19bd20a1931eb3a844fe in incubator-airflow's branch refs/heads/master
from [~bolke]
[;h=4764646 ]

[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun

_change_state_for_tis_without_dagrun was locking a
amount of tasks uncessarily. This could end up in
a deadlock
in the database due to the time the lock stood.

Closes #2267 from bolkedebruin/fix_deadlock

> _change_state_for_tis_without_dagrun can take very long and end up in deadlocks
> -------------------------------------------------------------------------------
>                 Key: AIRFLOW-1166
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.8.1
>            Reporter: Bolke de Bruin
>            Assignee: Bolke de Bruin
>              Labels: scheduler
>             Fix For: 1.8.2
> While testing multi-scheduler setups we've seen some serious delays (>8s, with just
one dag running) in _change_state_for_tis_without_dagrun. This can also result in deadlocks
in the db due to the fact the query in _change_state_for_tis_without_dagrun is locking a significant
amount of task instances. This is unnecessary.

This message was sent by Atlassian JIRA

View raw message