airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giacomo Tagliabe (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-63) Dangling Running Jobs
Date Fri, 06 May 2016 21:39:13 GMT
Giacomo Tagliabe created AIRFLOW-63:

             Summary: Dangling Running Jobs
                 Key: AIRFLOW-63
             Project: Apache Airflow
          Issue Type: Bug
          Components: scheduler
    Affects Versions: Airflow 1.7.0
         Environment: mac os X with local executor
            Reporter: Giacomo Tagliabe
            Priority: Minor

It seems that if the scheduler is killed unexpectedly, the SchedulerJob remains marked as
running. Same thing applies to LocalTaskJob: if a job is running when the scheduler dies,
the job remains marked as running forever. I'd expect `kill_zombies` to mark the job with
an old heartbeat as not running, but it seems it only marks the related task instances. This
to me seems like a bug, I also fail to see the piece of code that  is supposed to do that,
which leads me to think that this is not handled at all. I don't think there is anything really
critical about having stale jobs marked as running, but they definitely is confusing to see

This message was sent by Atlassian JIRA

View raw message