airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Davydov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-1980) Scheduler DB Queries Should be Batched
Date Tue, 09 Jan 2018 23:33:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dan Davydov updated AIRFLOW-1980:
---------------------------------
    Description: 
DB queries in the scheduler should be batched, since the larger ones can get quite big and
cause lock wait timeouts.

One example is the queries in the _change_state_for_tis_without_dagrun function in jobs.py

The orphan task cleanup is another such piece of code.

  was:
DB queries in the scheduler should be batched, since the larger ones can get quite big and
cause lock wait timeouts.

One example is the queries in the _change_state_for_tis_without_dagrun function in jobs.py


> Scheduler DB Queries Should be Batched
> --------------------------------------
>
>                 Key: AIRFLOW-1980
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1980
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: Dan Davydov
>
> DB queries in the scheduler should be batched, since the larger ones can get quite big
and cause lock wait timeouts.
> One example is the queries in the _change_state_for_tis_without_dagrun function in jobs.py
> The orphan task cleanup is another such piece of code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message