airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anton Erholt <anton.erh...@epidemicsound.com>
Subject Re: Stopping / killing dag runs gracefully
Date Fri, 22 Jan 2021 09:29:20 GMT
Apologies for not answering your question about how to stop DAG runs. I do
not know that. However, I wonder if it would make sense to pass along how
long the job should as a parameter to the optimization task, and when it
times out, exit/return appopriately so Airflow can read it?

Best,
Anton

Den fre 22 jan. 2021 08:01Franco Peschiera <franco.peschiera@gmail.com>
skrev:

> Hello again everyone,
>
> As the title says: I would like to be able to stop / kill a dagrun. I’ve
> seen this question asked here
> <https://stackoverflow.com/questions/43631693/how-to-stop-kill-airflow-tasks-from-the-ui>
> and here
> <https://stackoverflow.com/questions/49039386/how-do-i-stop-an-airflow-dag>
> .
> Several solutions are proposed but I was wondering if there is a “correct”
> way to stop a dagrun. I’m guessing from the airflow 2.0 docs of the REST
> API that it is probably not possible from there (since I did not see that).
> And since I do not see it anywhere in the docs, I fear there may not be a
> good way to do that properly. Is that so? Is there even an “improper” way?
>
> As context, the tasks that we want to schedule are optimization tasks that
> do not have a fixed time to run. Users usually put a time limit, e.g., an
> hour, and we would anyway put one by default if they don’t. But, in
> general, users may want to stop an execution if they see it takes too long
> or if they want to change something before running it again. So scheduling
> and stopping dagruns should be a “common” thing to do.
>
> Thanks as always!
>
> Franco
>

Mime
View raw message