airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1345) Don't commit on each loop
Date Fri, 14 Jul 2017 23:39:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088294#comment-16088294
] 

ASF subversion and git services commented on AIRFLOW-1345:
----------------------------------------------------------

Commit 0dd00291d74e10a30ed328c8542416b78e24bc06 in incubator-airflow's branch refs/heads/master
from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=0dd0029 ]

[AIRFLOW-1345] Dont expire TIs on each scheduler loop

TIs get expired on commit, which causes any access
to their properties
to cause a new query to the DB to be issued,
causing an n+1 query issue,
even when the TI is not scheduled. This change
makes all queries
batches, which will make scheduling substantially
faster.

Closes #2397 from saguziel/aguziel-commit-last


> Don't commit on each loop
> -------------------------
>
>                 Key: AIRFLOW-1345
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1345
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>             Fix For: 1.9.0
>
>
> RIght now, in the main scheduler loop, we commit for each TI. While this minimize the
time is a lock held, this expires all TIs, forcing us to do an n+1 query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message