airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Költringer (JIRA) <>
Subject [jira] [Commented] (AIRFLOW-2319) Table "dag_run" has (bad) second index on (dag_id, execution_date)
Date Tue, 04 Sep 2018 21:22:00 GMT


Andreas Költringer commented on AIRFLOW-2319:

I was thinking about possible fixes (see my comment above). The problem are the different
database backends - e.g. Sqlite does not support dropping uniqueness constraints.

Lacking confirmation by the projects' top committers/leaders that this is actually a bug (and
not intended due to some reason I might not see) - I did not proceed.

> Table "dag_run" has (bad) second index on (dag_id, execution_date)
> ------------------------------------------------------------------
>                 Key: AIRFLOW-2319
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DagRun
>    Affects Versions: 1.9.0
>            Reporter: Andreas Költringer
>            Priority: Major
> Inserting DagRun's via {{airflow.api.common.experimental.trigger_dag}} (multiple rows
with the same {{(dag_id, execution_date)}}) raised the following error:
> {code:java}
> {} ERROR - No row was found for one(){code}
> This is weird as the {{session.add()}} and {{session.commit()}} is right before {{run.refresh_from_db()}}
in {{models.DAG.create_dagrun()}}.
> Manually inspecting the database revealed that there is an extra index with {{unique}}
constraint on the columns {{(dag_id, execution_date)}}:
> {code:java}
> sqlite> .schema dag_run
> CREATE TABLE dag_run (
>         id INTEGER NOT NULL, 
>         dag_id VARCHAR(250), 
>         execution_date DATETIME, 
>         state VARCHAR(50), 
>         run_id VARCHAR(250), 
>         external_trigger BOOLEAN, conf BLOB, end_date DATETIME, start_date DATETIME,

>         PRIMARY KEY (id), 
>         UNIQUE (dag_id, execution_date), 
>         UNIQUE (dag_id, run_id), 
>         CHECK (external_trigger IN (0, 1))
> );
> CREATE INDEX dag_id_state ON dag_run (dag_id, state);{code}
> (On SQLite its a unique constraint, on MariaDB its also an index)
> The {{DagRun}} class in {{}} does not reflect this, however it is in [migrations/versions/|]
> I looked for other migrations correting this, but could not find any. As this is not
reflected in the model, I guess this is a bug?

This message was sent by Atlassian JIRA

View raw message