airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bo...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-950] Missing AWS integrations on documentation::integrations
Date Mon, 11 Sep 2017 13:03:48 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 728817f4c -> 01be025c5


[AIRFLOW-950] Missing AWS integrations on documentation::integrations

Closes #2552 from Swalloow/master


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/01be025c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/01be025c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/01be025c

Branch: refs/heads/master
Commit: 01be025c5e76bba2102d180e90a5d1ae8f398574
Parents: 728817f
Author: Swalloow <joon920570@gmail.com>
Authored: Mon Sep 11 15:03:40 2017 +0200
Committer: Bolke de Bruin <bolke@xs4all.nl>
Committed: Mon Sep 11 15:03:40 2017 +0200

----------------------------------------------------------------------
 docs/integration.rst | 96 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 94 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/01be025c/docs/integration.rst
----------------------------------------------------------------------
diff --git a/docs/integration.rst b/docs/integration.rst
index 99311d0..3b50586 100644
--- a/docs/integration.rst
+++ b/docs/integration.rst
@@ -60,10 +60,102 @@ WasbHook
 
 .. _AWS:
 
-AWS: Amazon Webservices
+AWS: Amazon Web Services
 -----------------------
 
-To be filled.
+Airflow has extensive support for Amazon Web Services. But note that the Hooks, Sensors and
+Operators are in the contrib section.
+
+AWS EMR
+''''''''
+
+- :ref:`EmrAddStepsOperator` : Adds steps to an existing EMR JobFlow.
+- :ref:`EmrCreateJobFlowOperator` : Creates an EMR JobFlow, reading the config from the EMR
connection.
+- :ref:`EmrTerminateJobFlowOperator` : Terminates an EMR JobFlow.
+- :ref:`EmrHook` : Interact with AWS EMR.
+
+.. _EmrAddStepsOperator:
+
+EmrAddStepsOperator
+""""""""
+
+.. autoclass:: airflow.contrib.operators.emr_add_steps_operator.EmrAddStepsOperator
+
+.. _EmrCreateJobFlowOperator:
+
+EmrCreateJobFlowOperator
+""""""""
+
+.. autoclass:: airflow.contrib.operators.emr_create_job_flow_operator.EmrCreateJobFlowOperator
+
+.. _EmrTerminateJobFlowOperator:
+
+EmrTerminateJobFlowOperator
+""""""""
+
+.. autoclass:: airflow.contrib.operators.emr_terminate_job_flow_operator.EmrTerminateJobFlowOperator
+
+.. _EmrHook:
+
+EmrHook
+""""""""
+
+.. autoclass:: airflow.contrib.hooks.emr_hook.EmrHook
+
+
+AWS S3
+'''''''
+
+- :ref:`S3FileTransformOperator` : Copies data from a source S3 location to a temporary location
on the local filesystem.
+- :ref:`S3ToHiveTransfer` : Moves data from S3 to Hive. The operator downloads a file from
S3, stores the file locally before loading it into a Hive table.
+- :ref:`S3Hook` : Interact with AWS S3.
+
+.. _S3FileTransformOperator:
+
+S3FileTransformOperator
+""""""""""""""""""""""""
+
+.. autoclass:: airflow.operators.s3_file_transform_operator.S3FileTransformOperator
+
+.. _S3ToHiveTransfer:
+
+S3ToHiveTransfer
+"""""""""""""""""
+
+.. autoclass:: airflow.operators.s3_to_hive_operator.S3ToHiveTransfer
+
+.. _S3Hook:
+
+S3Hook
+"""""""
+
+.. autoclass:: airflow.hooks.S3_hook.S3Hook
+
+
+AWS EC2 Container Service
+''''''''''''''''''''''''''
+
+- :ref:`ECSOperator` : Execute a task on AWS EC2 Container Service.
+
+.. _ECSOperator:
+
+ECSOperator
+""""""""""""
+
+.. autoclass:: airflow.contrib.operators.ecs_operator.ECSOperator
+
+
+AWS RedShift
+'''''''''''''
+
+- :ref:`RedshiftToS3Transfer` : Executes an unload command to S3 as a CSV with headers.
+
+.. _RedshiftToS3Transfer:
+
+RedshiftToS3Transfer
+"""""""""""""""""""""
+
+.. autoclass:: airflow.operators.redshift_to_s3_operator.RedshiftToS3Transfer
 
 
 


Mime
View raw message