airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Fei <cfe...@gmail.com>
Subject 1.8.0 Backfill Clarification
Date Tue, 18 Apr 2017 22:45:30 GMT
Hi all,



I'm new to Airflow, and I'm looking for someone to clarify the expected
behavior of running a backfill with regard to previously successful
tasks. When I run a backfill on 1.8.0, tasks that were previously run
successfully are re-run for me. Is it expected that backfills re-run all
tasks, even those that were marked as successful? For reference, the
command I'm running is `airflow backfill -s 2017-04-01 -e 2017-04-03
Tutorial`.


I wasn't able to find anything in the documentation to indicate either
which way. Some brief research revealed that invoking backfill was meant
at one point to "fill in the blanks", which I interpret to mean "only
run tasks that were not completed successfully". On the contrary, the
code *does* seem to explicitly set all task instances for a given DAGRun
to SCHEDULED (see [AIRFLOW-910][1] and
https://github.com/apache/incubator-airflow/pull/2107/files#diff-54a57ccc2c8e73d12c812798bf79ccb2R1816).


Apologies for such a fundamental question, just want to make sure I'm
not missing something obvious here. Can someone clarify?


Thanks,

Chris Fei


Links:

  1. https://issues.apache.org/jira/browse/AIRFLOW-910

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message