airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kaxil Naik (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-2279) Clearing Tasks Across DAGs
Date Fri, 10 Jan 2020 01:45:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kaxil Naik resolved AIRFLOW-2279.
---------------------------------
    Fix Version/s: 2.0.0
       Resolution: Fixed

> Clearing Tasks Across DAGs
> --------------------------
>
>                 Key: AIRFLOW-2279
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2279
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Achal Soni
>            Assignee: Qian Yu
>            Priority: Major
>             Fix For: 2.0.0
>
>         Attachments: cross_dag_ui_screenshot.png
>
>
> At Stripe, we commonly have discrete dags that depend on each other by leveraging ExternalTaskSensors.
We also find ourselves routinely wanting to not only clear tasks and their downstream tasks
in a particular dag, but also their downstream tasks in their dependent dags (linked by ExternalTaskSensors). 
> We currently have extended Airflow to handle this by modifying the webapp and cli tool
to optionally clear dependent tasks across multiple dags (see attached screenshot). 
> We want to open the floor for discussion with the larger Airflow community about the
usage of ExternalTaskSensors and specifically how to handle clearing across dags. We are interested
in learning more about the accepted practices in this regard, and are very open/willing to
contribute in this area if there is interest!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message