airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-467) BigQuery O+H should be able to work with other project then defined in connection
Date Sun, 04 Sep 2016 13:11:20 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15462909#comment-15462909
] 

ASF subversion and git services commented on AIRFLOW-467:
---------------------------------------------------------

Commit c6cc01f4e9542150c4bb9717e75d947e130419a8 in incubator-airflow's branch refs/heads/master
from [~alexvanboxel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=c6cc01f ]

[AIRFLOW-467] Allow defining of project_id in BigQueryHook

Introduced a toplevel table splitter that also
allows for defining a project_id as a suffix seperated by
the legacy colon or the new dot. If the project is not
defined, the default project_id will be used.

As it's so common of all the BigQuery classes
(Hook, Cursor, ...) the same splitter is used over all
those classes.

The documentation is adapted to allow defining the
suffix seperated as with the legacy colon as well
as with the new SQL dotted notation.

The change is 100% backwards compatible. Unit
tests are added for all scenario's, including negative
and compatibility.

Closes #1781 from alexvanboxel/feature/bq_tablename


> BigQuery O+H should be able to work with other project then defined in connection
> ---------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-467
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-467
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: gcp
>            Reporter: Alex Van Boxel
>            Assignee: Alex Van Boxel
>
> The BigQuery operators are not able to work with project other then the main project.
We should be able prefix the dataset+tablename with the project name.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message