airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-173) Create a FileSensor / NFSFileSensor sensor
Date Wed, 15 Jun 2016 20:03:09 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332445#comment-15332445
] 

ASF subversion and git services commented on AIRFLOW-173:
---------------------------------------------------------

Commit 06e70e2d13c47e1e4bbc4a9146bc93a02b45f98c in incubator-airflow's branch refs/heads/master
from Andre F de Miranda
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=06e70e2 ]

[AIRFLOW-173] Initial implementation of FileSensor

Implement a simple FileSensor to detect the creation of files and folder
in a given filesystem (a la HDFSSensor)

Closes #1543 from trixpan/AIRFLOW-173


> Create a FileSensor / NFSFileSensor sensor
> ------------------------------------------
>
>                 Key: AIRFLOW-173
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-173
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Andre
>            Priority: Minor
>
> While HDFS and WebHDFS suit most organisations using Hadoop, for some shops running MapR-FS,
Airflow implementation is simplified by the use of plain files pointing to MapR's NFS gateways.
> A FileSensor and/or a NFSFileSensor would assist the adoption of Airflow within the MapR
customer base, but more importantly, help those who are using POSIX compliant distributed
filesystems that can be mounted on Unix derivative systems (e.g. as MapR-FS (via NFS), CephFS,
GlusterFS, etc).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message