Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Sun, 22 Jan 2017 12:21:26 +0000 (UTC)
From: "Bolke de Bruin (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13036888.1485087669000.73096.1485087686363@Atlassian.JIRA>
In-Reply-To: <JIRA.13036888.1485087669000@Atlassian.JIRA>
References: <JIRA.13036888.1485087669000@Atlassian.JIRA> <JIRA.13036888.1485087669473@jira-lw-us.apache.org>
Subject: [jira] [Created] (AIRFLOW-791) At start up all running dag_runs are
 being checked, but not fixed
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sun, 22 Jan 2017 12:21:34 -0000

Bolke de Bruin created AIRFLOW-791:
--------------------------------------

             Summary: At start up all running dag_runs are being checked, b=
ut not fixed
                 Key: AIRFLOW-791
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-791
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Bolke de Bruin


At start up all running dag_runs are being checked, we seemed to have a lot=
 of =E2=80=9Cleft over=E2=80=9D dag_runs (couple of thousand)
- Checking was logged to INFO -> requires a fsync for every log message mak=
ing it very slow
- Checking would happen at every restart, but dag_runs=E2=80=99 states were=
 not being updated
- These dag_runs would never er be marked anything else than running for so=
me reason
-> Applied work around to update all dag_run in sql before a certain date t=
o -> finished
-> need to investigate why dag_runs did not get marked =E2=80=9Cfinished/fa=
iled=E2=80=9D=20


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)