airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ash Berlin-Taylor (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-63) Dangling Running Jobs
Date Mon, 22 Oct 2018 09:13:00 GMT


Ash Berlin-Taylor commented on AIRFLOW-63:

Possibly, though if the scheduler process is killed hard (oom, segfault etc) there still may
be cases where the job remains running. So I think I'd say "not quite yet" and this is still
possibly an issue (at least not fixed by my PR)

> Dangling Running Jobs
> ---------------------
>                 Key: AIRFLOW-63
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.7.0
>         Environment: mac os X with local executor
>            Reporter: Giacomo Tagliabe
>            Priority: Minor
> It seems that if the scheduler is killed unexpectedly, the SchedulerJob remains marked
as running. Same thing applies to LocalTaskJob: if a job is running when the scheduler dies,
the job remains marked as running forever. I'd expect `kill_zombies` to mark the job with
an old heartbeat as not running, but it seems it only marks the related task instances. This
to me seems like a bug, I also fail to see the piece of code that  is supposed to do that,
which leads me to think that this is not handled at all. I don't think there is anything really
critical about having stale jobs marked as running, but they definitely is confusing to see

This message was sent by Atlassian JIRA

View raw message