airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ash Berlin-Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-3050) Deactivate Stale DAGs in the Main Scheduler Loop
Date Wed, 12 Sep 2018 19:54:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612658#comment-16612658
] 

Ash Berlin-Taylor commented on AIRFLOW-3050:
--------------------------------------------

I would probably say at scheduler start up and then once every X (where X is minutes, or loops
around the scheduler or something) - running it every time around the scheduling loops sounds
too much

> Deactivate Stale DAGs in the Main Scheduler Loop
> ------------------------------------------------
>
>                 Key: AIRFLOW-3050
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3050
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>            Priority: Major
>
> Currently there is logic to deactivate stale DAGs in the scheduler, but it is only run
after the scheduler completes. Since most companies run the scheduler in a continuous loop,
this code never runs. This code should be moved into the main scheduler loop.
>  
> This would largely obviate the UI/API endpoints for deleting DAGs.
>  
> Mailing List Thread On This Topic:
> https://lists.apache.org/thread.html/8ac996a69bfa8214e5f56f2851cea349e0d83dcbb0d9e5af010525b9@%3Cdev.airflow.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message