airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ash Berlin-Taylor <...@apache.org>
Subject Re: Table "dag_run" has unique constraint on (dag_id, execution_date)
Date Fri, 18 Jan 2019 19:20:03 GMT
So that it's here on a searchable record:

Simply this is how airflow identifies tasks - `airflow run my_dag_id 2018-01-17 my task`.

If you were to have two dag runs for the exact same milliseconds which one would it run? Much
of the system treats this combo as unique, so lots of code would need to change to make it
not.

But: what is your use case here? The same dag running twice at for the same period doesn't
make sense to me conceptual (but then I admit I now think in terms of what airflow does internally
do my thinking is coloured)

-ash

On 18 January 2019 19:14:01 GMT, "Andreas K├Âltringer" <andreas.koeltringer@n-fuse.co>
wrote:
>Hi,
>
>almost a year ago I reported an issue to Airflow's Jira:
>https://issues.apache.org/jira/browse/AIRFLOW-2319
>
>Recently, someone pointed out that I should ask here on the mailing
>list (I 
>thought I did, but apparently, I did not).
>
>So here's the thing: the DagRun table has a unique constraint on
>(dag_id, 
>exec_date), that was not reflected by models.py (now
>models/__init__.py). This 
>got "fixed" in June 2018 - apparently this was not the only place where
>model 
>declaration and schema migration scripts were out of sync.
>
>Ash now pointed out on AIRFLOW-2319, that the unique constraint is
>there "by 
>design", meaning there's a reason why it's there. Can somebody please
>explain 
>to me what's the reasoning behind this? For us it would be a use case
>to have 
>multiple dag runs per (dag_id, exec_date).
>
>thx in advance!
>
>-- 
>Andreas Koeltringer
>Mail:   andreas.koeltringer@n-fuse.co
>
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message