airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tahir Fayyaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-732) Unable to upload AVRO file from GCS to BigQuery with no schema
Date Thu, 05 Jan 2017 12:50:58 GMT
Tahir Fayyaz created AIRFLOW-732:
------------------------------------

             Summary: Unable to upload AVRO file from GCS to BigQuery with no schema
                 Key: AIRFLOW-732
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-732
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
    Affects Versions: Airflow 1.8, Airflow 1.7.1.3
            Reporter: Tahir Fayyaz
            Priority: Minor
             Fix For: Airflow 1.8, Airflow 1.7.1.3


For the gcs_to_bq the operator you have to provide a schema and the fails if you do not provide
a schema.

However for AVRO uploads you do not need to provide a schema as this optional.

Currently there is no schema is provided None is actually passed through which causes the
job to fail for AVRO uploads.

You can also actually tell the able to autodetect for CSV and JSON also using (configuration.load.autodetect)
- more details https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message