airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henk Griffioen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-1065) Add functionality for Azure Blob Storage
Date Tue, 04 Apr 2017 08:23:41 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Henk Griffioen updated AIRFLOW-1065:
------------------------------------
    Description: 
Currently Airflow has sensors and operators for S3 and GCE but it does not support Azure Blob
Storage.

A hook would interface with Azure Blob storage via the Python library azure-storage over the
wasb protocol. Sensors use the hook to detect if a blob has landed on a container and operators
use it to move files to the blob storage.

The design for the hook airflow.contrib.hooks.WasbHook would mimic airflow.operators.S3_hook.S3Hook.

  was:
Currently Airflow has sensors and operators for S3 and GCE but it does not support Azure Blob
Storage.

A hook would interface with Azure Blob storage via the Python library azure-storage over the
wasbs protocol. Sensors use the hook to detect if a blob has landed on a container and operators
use it to move files to the blob storage.

The design for the hook airflow.contrib.hooks.WasbsHook would mimic airflow.operators.S3_hook.S3Hook.


> Add functionality for Azure Blob Storage
> ----------------------------------------
>
>                 Key: AIRFLOW-1065
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1065
>             Project: Apache Airflow
>          Issue Type: New Feature
>            Reporter: Henk Griffioen
>
> Currently Airflow has sensors and operators for S3 and GCE but it does not support Azure
Blob Storage.
> A hook would interface with Azure Blob storage via the Python library azure-storage over
the wasb protocol. Sensors use the hook to detect if a blob has landed on a container and
operators use it to move files to the blob storage.
> The design for the hook airflow.contrib.hooks.WasbHook would mimic airflow.operators.S3_hook.S3Hook.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message