airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From twinkle sachdeva <twinkle.sachd...@gmail.com>
Subject Re: Airflow state change diagram
Date Wed, 02 Nov 2016 11:43:46 GMT
Thanks Gerard for sharing it.

Regards,
Twinkle

On Mon, Oct 31, 2016 at 2:35 AM, Gerard Toonstra <gtoonstra@gmail.com>
wrote:

> I was looking at trying to fix AIRFLOW-137 (max_active_runs not respected),
> but quickly noticed that the code that does all the scheduling is rather
> complex with state updates going on across multiple source files in
> multiple threads, etc.
>
> It's then best to find a suitable way to visualize all this complexity, so
> I built this state change diagram:
>
> https://docs.google.com/spreadsheets/d/1vVvOwfDSacTC_YzwUkOMyykP6LiipCeoW_
> V70PuFrN4/edit?usp=sharing
>
> The state changes represent a potential execution path where the state for
> a task instance will be updated to that value. Backfill is not considered
> in this diagram. States for dagruns/jobs/dags are also not considered.
>
> Could be useful for someone else.
>
> Rgds,
>
> Gerard
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message