airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Felix Yuan (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1786) Upstream Sensor skipped but whole DAG fails
Date Tue, 07 Nov 2017 02:43:00 GMT
Felix Yuan created AIRFLOW-1786:

             Summary: Upstream Sensor skipped but whole DAG fails
                 Key: AIRFLOW-1786
             Project: Apache Airflow
          Issue Type: Bug
          Components: DAG
    Affects Versions: Airflow 1.8
            Reporter: Felix Yuan
            Priority: Minor

I have created a daily DAG, which first monitor whether a file is ready on FTP server, then
execute an ETL operation. If someday the file is missing, this is expected, so I set the "soft_fail"
as True to simply skipped the FTP sensor.

During execution, I see the task of "FTP sensor" is surely be skipped, but the whole DAG is
marked as failed.

My expectation is the upstream task (FTP Sensor) is skipped, then all downstream tasks are
skipped as well.

This message was sent by Atlassian JIRA

View raw message