airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Iuliia Volkova (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-540) Allow BigQuery operators to specify default dataset
Date Tue, 11 Sep 2018 19:22:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611128#comment-16611128
] 

Iuliia Volkova commented on AIRFLOW-540:
----------------------------------------

Kaxil Naik , what do you think about this task? Now into the bigquery_hook was putted logic,
what target table must always be provided with the dataset, I talk about https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L311

> Allow BigQuery operators to specify default dataset
> ---------------------------------------------------
>
>                 Key: AIRFLOW-540
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-540
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib
>            Reporter: Sam McVeety
>            Priority: Minor
>
> BigQuery allows the specification of a defaultDataset (https://cloud.google.com/bigquery/docs/reference/v2/jobs/query),
which can simplify and modularize the SQL being used.
> Perhaps a more general solution to this (and https://issues.apache.org/jira/browse/AIRFLOW-539)
would be to support arbitrary kwargs to the various BigQueryHook methods.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message