airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jack (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRFLOW-4718) FileSensor does not distinguish non-existent file from other IO failures
Date Sun, 04 Aug 2019 11:33:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899611#comment-16899611
] 

jack edited comment on AIRFLOW-4718 at 8/4/19 11:32 AM:
--------------------------------------------------------

Can you raise PR for this?

Seems like same issue as https://issues.apache.org/jira/browse/AIRFLOW-1450


was (Author: jackjack10):
Can you raise PR for this?

> FileSensor does not distinguish non-existent file from other IO failures
> ------------------------------------------------------------------------
>
>                 Key: AIRFLOW-4718
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4718
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: 1.10.3
>            Reporter: Kirill Zhuravlev
>            Priority: Minor
>
> airflow.contrib.sensors.file_sensor.FileSensor does not distinguish non-existent file
from other IO failures (e.g. wrong permissions, etc.)
> This limits its uses in scenarios with `soft_fail=True`, where it can mistake a different
IOError for missing file and skip the rest of the workflow.
> Example case:
>  * file is present in the fs
>  * it (its parent folder) doesn't have listing permissions for airflow user
>  * `soft_fail=True` for the sensor
>  * sensor skips the workflow despite the file being there
> Expected behavior:
>  * failed task in case of any exception other than IOError with errno=={{errno.}}{{ENOENT}}
(==2)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message