airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bolke de Bruin (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
Date Sat, 18 Mar 2017 00:34:41 GMT


Bolke de Bruin commented on AIRFLOW-992:

That would make sense I think. Have you tested it yet that way?

On a side note: Would you mind providing your edits as comments? It is difficult to find out
what has changed if reading from a phone.

> Skipped tasks do not propagate correctly
> ----------------------------------------
>                 Key: AIRFLOW-992
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Dan Davydov
>            Priority: Critical
> We are seeing skipped tasks not being propagated correctly:
> E.g. 
> A->B
> `-->C
> Task A depends on task B and C
> If B gets skipped and C gets run then:
> Expected:
> A will get skipped
> EDIT: Upon further investigation this was caused by a change in the semantics of ALL_SUCCESS,
which I have these feelings about:
> Intuitively you would expect to skip any task that has dependencies that weren't run
by default, i.e. the trigger rule is called ALL_SUCCESS and skipped tasks are not successful
ones, and that was also the old behavior in 1.7.3.
> This is going to break some use cases which could be alright, but I feel these new semantics
make less sense than before so it's a bad reason to break existing use cases.
> I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but again I
feel this is hacky and really we should have the old ALL_SUCCESS (default) and a new ALL_SUCCESS_OR_SKIPPED
trigger rule if desired.
> Actual:
> A gets run
> It's possible the reason for the change in semantics (which was to fix a bug) is fixed
already by and the only thing we need
to do is revert
> [~bolke]

This message was sent by Atlassian JIRA

View raw message