airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weiwei Zhang <vvianzh...@gmail.com>
Subject Re: airflow backfill seems to ignore -I
Date Wed, 05 Jul 2017 15:45:43 GMT
I am using airflow 1.8.1 as well. It is able to pick up the rest of the tasks when using backfill
with the only exception which is when there is a task failed and I had to clear the status
to allow the backfill to work. Any ideas why it is behaving like this? The previous version
1.6.2 didn't require clearing the failed task before doing backfill.

Thx a lot,
Viv

> On Jul 5, 2017, at 7:38 AM, Tobias Feldhaus <Tobias.Feldhaus@localsearch.ch> wrote:
> 
> I’ve just pulled the newest master and build it; the behaviour is the same. How can
it be that “–i” is not honoured and dependencies are checked?
> 
> 
> On 05.07.2017, 15:49, "Tobias Feldhaus" <Tobias.Feldhaus@localsearch.ch> wrote:
> 
>    But nonetheless, is it not possible to backfill and ignore the upstream dependencies
with “-i” ?
> 
>    On 05.07.2017, 14:34, "Tobias Feldhaus" <Tobias.Feldhaus@localsearch.ch> wrote:
> 
>        I meant –i , but I just needed to manually set the upstream things to success
and it worked. Nevermind.
> 
>        Best,
>        Tobi
> 
>        On 05.07.2017, 14:28, "Tobias Feldhaus" <Tobias.Feldhaus@localsearch.ch<mailto:Tobias.Feldhaus@localsearch.ch>>
wrote:
> 
>        Hi,
> 
>        When running airflow (1.8.1) backfill with –I and –t like:
> 
>        airflow backfill -t 'nonspider_sessions' -i -I -s 2017-05-30 -e 2017-05-31 google_pipelines
> 
>        I would expect it to rerun that specific task and ignoring the dependencies. Instead
I see this:
> 
>        [2017-07-05 12:23:30,419] {base_task_runner.py:95} INFO - Subtask: [2017-07-05
12:23:30,419] {models.py:1145} INFO - Dependencies not met for <TaskInstance: google_pipelines.nonspider_sessions
2017-05-30 05:30:00 [queued]>, dependency 'Trigger Rule' FAILED: Task's trigger rule 'all_success'
requires all upstream tasks to have succeeded, but found 3 non-success(es). upstream_tasks_state={'successes':
0L, 'failed': 0L, 'upstream_failed': 0L, 'skipped': 0L, 'done': 0L}, upstream_task_ids=['frontend_sensor',
'log_sensor', 'tracker_pipeline']
> 
>        Am I doing it wrong?
> 
> 
> 
>        Best,
>        Tobi
> 
> 
> 
> 

Mime
View raw message