airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: "intra-day" backfill mistake
Date Tue, 31 May 2016 16:05:37 GMT
I agree. The date format handling in the CLI is not intuitive. IMO, it
should force full-resolution YYYYMMDDHHMMSS.micro.

On Fri, May 27, 2016 at 7:28 PM, Bence Nagy <bence@underyx.me> wrote:

> I can think of weird legit reasons why one would force running the dag on
> an intra-day start date, but a warning message requiring confirmation
> (perhaps also offering to autocorrect the date to the nearest schedule
> matching one) would be awesome to have.
>
> On Sat, May 28, 2016, 3:22 AM Lance Norskog <lance.norskog@gmail.com>
> wrote:
>
> > I just made the same mistake, twice.
> >
> > I have a Dag with this schedule:
> >
> > dag = DAG(
> >     dag_id='xyz', default_args=args,
> >     schedule_interval='0 2 * * *',
> >     start_date=datetime(2016, 5, 16,2),
> >     max_active_runs=1
> >     )
> >
> >
> > It runs daily, at 2AM UTC. I ran this command:
> > airflow backfill -s 2016-05-24 xyz
> >
> > So, of course, it created a new run at midnight UTC instead of running
> the
> > DAG at 2am.
> >
> > Should 'airflow backfill' and similar respect the periodicity of the DAG?
> > Could these command give an error and require a flag to force running
> > outside the DAG's periodicity?
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message