airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giovanni Briggs (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-638) Added schema_update_options to BigQueryBaseCursor and GoogleCloudStorageToBigQueryOperator
Date Fri, 18 Nov 2016 23:29:58 GMT
Giovanni Briggs created AIRFLOW-638:
---------------------------------------

             Summary: Added schema_update_options to BigQueryBaseCursor and GoogleCloudStorageToBigQueryOperator
                 Key: AIRFLOW-638
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-638
             Project: Apache Airflow
          Issue Type: New Feature
            Reporter: Giovanni Briggs
            Assignee: Giovanni Briggs
            Priority: Minor


Was trying to upload data to a BigQuery table along with a change in my schema.  The way that
the GoogleCloudStorageToBigQueryOperator currently works, this is not possible, but BigQuery
does support changing a table's schema as a side-effect of a load operation via the "schemaUpdateOptions"
field: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.schemaUpdateOptions

Added schema_update_options as a parameter to both the GoogleCloudStorageToBigQueryOperator
and consequently the BigQueryBaseCursor to enable use of the schemaUpdateOptions field during
load operations.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message