airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joy...@apache.org
Subject [1/2] incubator-airflow git commit: [AIRFLOW-2130] Add missing Operators to API Reference docs
Date Fri, 23 Feb 2018 15:31:12 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master ba126926d -> 846a13198


[AIRFLOW-2130] Add missing Operators to API Reference docs

* Fix autodoc import path for BaseSensorOperator
* Add autodoc imports for many core & contrib Operators that were missing
* Rename "Operator API" section to "Core Operators" for better contrast with the following
"Community-contributed Operators" section
* Add subheadings to API Reference#Operators. Since all the Sensors were already alphabetized
separately from the rest of the operators, this formalizes that distinction and moves all
the Transfer operators to their own section as well.
* Alphabetize Operator class names
* Improve formatting in top-level Operators section

This also fixes the earlier and more narrowly scoped [AIRFLOW-951]


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3e32819d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3e32819d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3e32819d

Branch: refs/heads/master
Commit: 3e32819d85acca4413d2d7d4bcd814c4c9af3241
Parents: ba12692
Author: Reid Beels <reid@thedyrt.com>
Authored: Tue Feb 20 12:18:46 2018 -0800
Committer: Reid Beels <reid@thedyrt.com>
Committed: Fri Feb 23 00:40:15 2018 -0800

----------------------------------------------------------------------
 docs/code.rst | 71 ++++++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 58 insertions(+), 13 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3e32819d/docs/code.rst
----------------------------------------------------------------------
diff --git a/docs/code.rst b/docs/code.rst
index 180e8da..5c1b000 100644
--- a/docs/code.rst
+++ b/docs/code.rst
@@ -4,8 +4,8 @@ API Reference
 Operators
 ---------
 Operators allow for generation of certain types of tasks that become nodes in
-the DAG when instantiated. All operators derive from BaseOperator and
-inherit many attributes and methods that way. Refer to the BaseOperator
+the DAG when instantiated. All operators derive from ``BaseOperator`` and
+inherit many attributes and methods that way. Refer to the BaseOperator_
 documentation for more details.
 
 There are 3 main types of operators:
@@ -37,37 +37,56 @@ All sensors are derived from ``BaseSensorOperator``. All sensors inherit
 the ``timeout`` and ``poke_interval`` on top of the ``BaseOperator``
 attributes.
 
-.. autoclass:: airflow.operators.base_sensor_operator.BaseSensorOperator
+.. autoclass:: airflow.sensors.base_sensor_operator.BaseSensorOperator
 
 
-Operator API
-''''''''''''
+Core Operators
+''''''''''''''
+
+Operators
+^^^^^^^^^
 
 .. autoclass:: airflow.operators.bash_operator.BashOperator
 .. autoclass:: airflow.operators.python_operator.BranchPythonOperator
-.. autoclass:: airflow.operators.dagrun_operator.TriggerDagRunOperator
+.. autoclass:: airflow.operators.check_operator.CheckOperator
 .. autoclass:: airflow.operators.docker_operator.DockerOperator
 .. autoclass:: airflow.operators.dummy_operator.DummyOperator
 .. autoclass:: airflow.operators.email_operator.EmailOperator
 .. autoclass:: airflow.operators.generic_transfer.GenericTransfer
-.. autoclass:: airflow.operators.hive_to_samba_operator.Hive2SambaOperator
-.. autoclass:: airflow.operators.hive_operator.HiveOperator
 .. autoclass:: airflow.operators.hive_to_druid.HiveToDruidTransfer
 .. autoclass:: airflow.operators.hive_to_mysql.HiveToMySqlTransfer
-.. autoclass:: airflow.operators.http_operator.SimpleHttpOperator
+.. autoclass:: airflow.operators.hive_to_samba_operator.Hive2SambaOperator
+.. autoclass:: airflow.operators.hive_operator.HiveOperator
+.. autoclass:: airflow.operators.hive_stats_operator.HiveStatsCollectionOperator
+.. autoclass:: airflow.operators.check_operator.IntervalCheckOperator
+.. autoclass:: airflow.operators.jdbc_operator.JdbcOperator
+.. autoclass:: airflow.operators.latest_only_operator.LatestOnlyOperator
 .. autoclass:: airflow.operators.mssql_operator.MsSqlOperator
 .. autoclass:: airflow.operators.mssql_to_hive.MsSqlToHiveTransfer
+.. autoclass:: airflow.operators.mysql_operator.MySqlOperator
+.. autoclass:: airflow.operators.mysql_to_hive.MySqlToHiveTransfer
+.. autoclass:: airflow.operators.oracle_operator.OracleOperator
+.. autoclass:: airflow.operators.pig_operator.PigOperator
 .. autoclass:: airflow.operators.postgres_operator.PostgresOperator
 .. autoclass:: airflow.operators.presto_check_operator.PrestoCheckOperator
 .. autoclass:: airflow.operators.presto_check_operator.PrestoIntervalCheckOperator
+.. autoclass:: airflow.operators.presto_to_mysql.PrestoToMySqlTransfer
 .. autoclass:: airflow.operators.presto_check_operator.PrestoValueCheckOperator
 .. autoclass:: airflow.operators.python_operator.PythonOperator
 .. autoclass:: airflow.operators.python_operator.PythonVirtualenvOperator
+.. autoclass:: airflow.operators.s3_file_transform_operator.S3FileTransformOperator
 .. autoclass:: airflow.operators.s3_to_hive_operator.S3ToHiveTransfer
-.. autoclass:: airflow.operators.ShortCircuitOperator
+.. autoclass:: airflow.operators.python_operator.ShortCircuitOperator
+.. autoclass:: airflow.operators.http_operator.SimpleHttpOperator
 .. autoclass:: airflow.operators.slack_operator.SlackAPIOperator
+.. autoclass:: airflow.operators.sqlite_operator.SqliteOperator
 .. autoclass:: airflow.operators.subdag_operator.SubDagOperator
+.. autoclass:: airflow.operators.dagrun_operator.TriggerDagRunOperator
+.. autoclass:: airflow.operators.check_operator.ValueCheckOperator
+.. autoclass:: airflow.operators.redshift_to_s3_operator.RedshiftToS3Transfer
 
+Sensors
+^^^^^^^
 .. autoclass:: airflow.sensors.external_task_sensor.ExternalTaskSensor
 .. autoclass:: airflow.sensors.hdfs_sensor.HdfsSensor
 .. autoclass:: airflow.sensors.hive_partition_sensor.HivePartitionSensor
@@ -84,6 +103,10 @@ Operator API
 Community-contributed Operators
 '''''''''''''''''''''''''''''''
 
+Operators
+^^^^^^^^^
+
+.. autoclass:: airflow.contrib.operators.awsbatch_operator.AWSBatchOperator
 .. autoclass:: airflow.contrib.operators.bigquery_check_operator.BigQueryCheckOperator
 .. autoclass:: airflow.contrib.operators.bigquery_check_operator.BigQueryValueCheckOperator
 .. autoclass:: airflow.contrib.operators.bigquery_check_operator.BigQueryIntervalCheckOperator
@@ -110,26 +133,48 @@ Community-contributed Operators
 .. autoclass:: airflow.contrib.operators.dataproc_operator.DataprocWorkflowTemplateInstantiateInlineOperator
 .. autoclass:: airflow.contrib.operators.datastore_export_operator.DatastoreExportOperator
 .. autoclass:: airflow.contrib.operators.datastore_import_operator.DatastoreImportOperator
+.. autoclass:: airflow.contrib.operators.druid_operator.DruidOperator
 .. autoclass:: airflow.contrib.operators.ecs_operator.ECSOperator
+.. autoclass:: airflow.contrib.operators.emr_add_steps_operator.EmrAddStepsOperator
+.. autoclass:: airflow.contrib.operators.emr_create_job_flow_operator.EmrCreateJobFlowOperator
+.. autoclass:: airflow.contrib.operators.emr_terminate_job_flow_operator.EmrTerminateJobFlowOperator
 .. autoclass:: airflow.contrib.operators.file_to_gcs.FileToGoogleCloudStorageOperator
 .. autoclass:: airflow.contrib.operators.file_to_wasb.FileToWasbOperator
 .. autoclass:: airflow.contrib.operators.gcs_copy_operator.GoogleCloudStorageCopyOperator
 .. autoclass:: airflow.contrib.operators.gcs_download_operator.GoogleCloudStorageDownloadOperator
 .. autoclass:: airflow.contrib.operators.gcs_list_operator.GoogleCloudStorageListOperator
 .. autoclass:: airflow.contrib.operators.gcs_operator.GoogleCloudStorageCreateBucketOperator
+.. autoclass:: airflow.contrib.operators.gcs_to_bq.GoogleCloudStorageToBigQueryOperator
 .. autoclass:: airflow.contrib.operators.gcs_to_gcs.GoogleCloudStorageToGoogleCloudStorageOperator
+.. autoclass:: airflow.contrib.operators.hipchat_operator.HipChatAPIOperator
+.. autoclass:: airflow.contrib.operators.hipchat_operator.HipChatAPISendRoomNotificationOperator
+.. autoclass:: airflow.contrib.operators.hive_to_dynamodb.HiveToDynamoDBTransferOperator
+.. autoclass:: airflow.contrib.operators.jira_operator.JiraOperator
+.. autoclass:: airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator
+.. autoclass:: airflow.contrib.operators.mlengine_operator.MLEngineBatchPredictionOperator
+.. autoclass:: airflow.contrib.operators.mlengine_operator.MLEngineModelOperator
+.. autoclass:: airflow.contrib.operators.mlengine_operator.MLEngineVersionOperator
+.. autoclass:: airflow.contrib.operators.mlengine_operator.MLEngineTrainingOperator
+.. autoclass:: airflow.contrib.operators.mysql_to_gcs.MySqlToGoogleCloudStorageOperator
+.. autoclass:: airflow.contrib.operators.postgres_to_gcs_operator.PostgresToGoogleCloudStorageOperator
 .. autoclass:: airflow.contrib.operators.pubsub_operator.PubSubTopicCreateOperator
 .. autoclass:: airflow.contrib.operators.pubsub_operator.PubSubTopicDeleteOperator
 .. autoclass:: airflow.contrib.operators.pubsub_operator.PubSubSubscriptionCreateOperator
 .. autoclass:: airflow.contrib.operators.pubsub_operator.PubSubSubscriptionDeleteOperator
 .. autoclass:: airflow.contrib.operators.pubsub_operator.PubSubPublishOperator
-.. autoclass:: airflow.contrib.operators.hipchat_operator.HipChatAPIOperator
-.. autoclass:: airflow.contrib.operators.hipchat_operator.HipChatAPISendRoomNotificationOperator
 .. autoclass:: airflow.contrib.operators.qubole_operator.QuboleOperator
+.. autoclass:: airflow.contrib.operators.sftp_operator.SFTPOperator
+.. autoclass:: airflow.contrib.operators.spark_jdbc_operator.SparkJDBCOperator
+.. autoclass:: airflow.contrib.operators.spark_sql_operator.SparkSqlOperator
+.. autoclass:: airflow.contrib.operators.spark_submit_operator.SparkSubmitOperator
+.. autoclass:: airflow.contrib.operators.sqoop_operator.SqoopOperator
 .. autoclass:: airflow.contrib.operators.ssh_operator.SSHOperator
 .. autoclass:: airflow.contrib.operators.vertica_operator.VerticaOperator
-.. autoclass:: airflow.contrib.operators.vertica_to_hive.VerticaToHiveTransfer
 .. autoclass:: airflow.contrib.operators.spark_submit_operator.SparkSubmitOperator
+.. autoclass:: airflow.contrib.operators.vertica_to_hive.VerticaToHiveTransfer
+
+Sensors
+^^^^^^^
 
 .. autoclass:: airflow.contrib.sensors.aws_redshift_cluster_sensor.AwsRedshiftClusterSensor
 .. autoclass:: airflow.contrib.sensors.bash_sensor.BashSensor


Mime
View raw message