airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Tallman <btall...@gmail.com>
Subject Re: Catchup By default = False vs LatestOnlyOperator
Date Sun, 22 Jul 2018 02:34:49 GMT
As the author of catch-up, the idea is that in many cases your data doesn't "window" nicely
and you want instead to just run as if it were a brilliant Cron...

Ben

Sent from my iPhone

> On Jul 20, 2018, at 11:39 PM, Shah Altaf <mendhak@gmail.com> wrote:
> 
> Hi my understanding is: if you use the LatestOnlyOperator then when you run
> the DAG for the first time you'll see a whole bunch of DAG runs queued up,
> and in each run the LatestOnlyOperator will cause the rest of the DAG run
> to be skipped.  Only the latest DAG will run in 'full'.
> 
> With catchup = False, you should just get just the latest DAG run.
> 
> 
> On Fri, Jul 20, 2018 at 10:58 PM Shubham Gupta <shubham180695.sg@gmail.com>
> wrote:
> 
>> ---------- Forwarded message ---------
>> From: Shubham Gupta <shubham180695.sg@gmail.com>
>> Date: Fri, Jul 20, 2018 at 2:38 PM
>> Subject: Catchup By default = False vs LatestOnlyOperator
>> To: <dev-subscribe@airflow.incubator.apache.org>
>> 
>> 
>> Hi!
>> 
>> Can someone please explain the difference b/w catchup by default = False
>> and LatestOnlyOperator?
>> 
>> Regarding
>> Shubham Gupta
>> 

Mime
View raw message