airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-1345) Don't commit on each loop
Date Fri, 14 Jul 2017 23:39:00 GMT


ASF subversion and git services commented on AIRFLOW-1345:

Commit 0dd00291d74e10a30ed328c8542416b78e24bc06 in incubator-airflow's branch refs/heads/master
from [~saguziel]
[;h=0dd0029 ]

[AIRFLOW-1345] Dont expire TIs on each scheduler loop

TIs get expired on commit, which causes any access
to their properties
to cause a new query to the DB to be issued,
causing an n+1 query issue,
even when the TI is not scheduled. This change
makes all queries
batches, which will make scheduling substantially

Closes #2397 from saguziel/aguziel-commit-last

> Don't commit on each loop
> -------------------------
>                 Key: AIRFLOW-1345
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>             Fix For: 1.9.0
> RIght now, in the main scheduler loop, we commit for each TI. While this minimize the
time is a lock held, this expires all TIs, forcing us to do an n+1 query.

This message was sent by Atlassian JIRA

View raw message