Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAD8uk9S5idZdyQGTukysd5sjVEVPOhga0nejQm-d7ASQ7GiUrw@mail.gmail.com>
References: <CAM5819Qas_heut5X-M9E4zrRS7=r_xYN6hWnHf4=EJ+_z0DeiQ@mail.gmail.com>
 <CAD8uk9S5idZdyQGTukysd5sjVEVPOhga0nejQm-d7ASQ7GiUrw@mail.gmail.com>
From: Joseph Napolitano <joseph.napolitano@blueapron.com.INVALID>
Date: Wed, 2 Nov 2016 11:07:15 -0400
Message-ID: <CAPp6otJd-aGoOvsV0aRreNkN2=G62f+We=1L4P3jqVMLzKgW5g@mail.gmail.com>
Subject: Re: Airflow state change diagram
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a11401936ef680e054052cb21
archived-at: Wed, 02 Nov 2016 15:07:39 -0000

--001a11401936ef680e054052cb21
Content-Type: text/plain; charset=UTF-8

Gerard,

This is great.  Thanks for sharing this.

Joe

On Wed, Nov 2, 2016 at 7:43 AM, twinkle sachdeva <twinkle.sachdeva@gmail.com
> wrote:

> Thanks Gerard for sharing it.
>
> Regards,
> Twinkle
>
> On Mon, Oct 31, 2016 at 2:35 AM, Gerard Toonstra <gtoonstra@gmail.com>
> wrote:
>
> > I was looking at trying to fix AIRFLOW-137 (max_active_runs not
> respected),
> > but quickly noticed that the code that does all the scheduling is rather
> > complex with state updates going on across multiple source files in
> > multiple threads, etc.
> >
> > It's then best to find a suitable way to visualize all this complexity,
> so
> > I built this state change diagram:
> >
> > https://docs.google.com/spreadsheets/d/1vVvOwfDSacTC_
> YzwUkOMyykP6LiipCeoW_
> > V70PuFrN4/edit?usp=sharing
> >
> > The state changes represent a potential execution path where the state
> for
> > a task instance will be updated to that value. Backfill is not considered
> > in this diagram. States for dagruns/jobs/dags are also not considered.
> >
> > Could be useful for someone else.
> >
> > Rgds,
> >
> > Gerard
> >
>


-- 
*Joe Napolitano *| Sr. Data Engineer
www.blueapron.com | 5 Crosby Street, New York, NY 10013

--001a11401936ef680e054052cb21--