airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1059) Reset_state_for_orphaned_task should operate in batch for the scheduler
Date Fri, 14 Jul 2017 23:38:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088290#comment-16088290
] 

ASF subversion and git services commented on AIRFLOW-1059:
----------------------------------------------------------

Commit e05d3b4df0013f0cff804dfbd1db0197f320de48 in incubator-airflow's branch refs/heads/master
from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=e05d3b4 ]

[AIRFLOW-1059] Reset orphaned tasks in batch for scheduler

The current implementation resets state for tasks
1 dagrun at a time. We
should be able to do this in larger batches, which
will improve
scheduler startup time.

Closes #2205 from saguziel/aguziel-reset-state


> Reset_state_for_orphaned_task should operate in batch for the scheduler
> -----------------------------------------------------------------------
>
>                 Key: AIRFLOW-1059
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1059
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>             Fix For: 1.9.0
>
>
> Scheduler startup is very slow due to resetting state making a query for each dag run.
We should be able to do this in a constant number of queries which will increase scheduler
startup time significantly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message