airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Kramer (Jira)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1242) BigQueryHook assumes that a valid project_id can't contain a ':'
Date Tue, 15 Oct 2019 23:07:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952363#comment-16952363
] 

Adam Kramer commented on AIRFLOW-1242:
--------------------------------------

[~zoy@google.com] I see that your test cases include a project of the form 'alt1:alt1' but
doesn't include a full domain (like alt1.com) that is appended for cross-domain project-ids
(e.g. 'foo.com:foo_project').  Was this a case you were trying to account for? I'm running
into this issue at the moment using a service account that has cross-domain project access. 

 

 

> BigQueryHook assumes that a valid project_id can't contain a ':'
> ----------------------------------------------------------------
>
>                 Key: AIRFLOW-1242
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1242
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: gcp
>            Reporter: Zohar Yahav
>            Assignee: Zohar Yahav
>            Priority: Minor
>             Fix For: 1.9.0
>
>
> BigQueryHook's _split_tablename method assumes that a project_id that contains a colon
is invalid, although projects can be named "<domain>:<project>".
> This means that such projects would have to rely on the default project_id being set,
instead of passing in "project_id:dataset.table"/"project_id.dataset.table".
> Code reference:
> https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L928



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message