airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Kulyk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-2923) LatestOnlyOperator cascade skip through all_done and dummy
Date Tue, 21 Aug 2018 06:55:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dmytro Kulyk updated AIRFLOW-2923:
----------------------------------
    Description: 
DAG with consolidating point (calc_ready : dummy)

as per [https://airflow.apache.org/concepts.html#latest-run-only] given task should be ran
even catching up an execution DagRuns for a previous periods
 However, LatestOnlyOperator cascading through calc_ready despite of it is a dummy and trigger_rule=all_done
 Same behavior when trigger_rule=all_success
{code}
t_ready = DummyOperator(
  task_id = 'calc_ready',
  trigger_rule = TriggerRule.ALL_DONE,
  dag=dag)
{code}
!screenshot-1.png!

  was:
DAG with consolidating point (calc_ready : dummy) 

as per https://airflow.apache.org/concepts.html#latest-run-only given task should be ran even
catching up an execution DagRuns for a previous periods
However, LatestOnlyOperator cascading through `calc_ready` despite of it is a `dummy` and
`trigger_rule`=`all_done`
Same behavior when `trigger_rule`=`all_success`

{code:python}
t_ready = DummyOperator(
  task_id = 'calc_ready',
  trigger_rule = TriggerRule.ALL_DONE,
  dag=dag)
{code} 

 !screenshot-1.png!


> LatestOnlyOperator cascade skip through all_done and dummy
> ----------------------------------------------------------
>
>                 Key: AIRFLOW-2923
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2923
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.9.0
>         Environment: CeleryExecutor, 2-nodes cluster, RMQ, PostgreSQL
>            Reporter: Dmytro Kulyk
>            Priority: Major
>              Labels: cascade, latestonly, skip
>         Attachments: cube_update.py, screenshot-1.png
>
>
> DAG with consolidating point (calc_ready : dummy)
> as per [https://airflow.apache.org/concepts.html#latest-run-only] given task should be
ran even catching up an execution DagRuns for a previous periods
>  However, LatestOnlyOperator cascading through calc_ready despite of it is a dummy and
trigger_rule=all_done
>  Same behavior when trigger_rule=all_success
> {code}
> t_ready = DummyOperator(
>   task_id = 'calc_ready',
>   trigger_rule = TriggerRule.ALL_DONE,
>   dag=dag)
> {code}
> !screenshot-1.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message