airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-509) Create operator to delete tables in BigQuery
Date Thu, 15 Sep 2016 17:18:20 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493995#comment-15493995
] 

ASF subversion and git services commented on AIRFLOW-509:
---------------------------------------------------------

Commit 0e3ed447b5ec854d4af918eeee19b9efee886918 in incubator-airflow's branch refs/heads/master
from [~illop]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=0e3ed44 ]

[AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery

We have a use case to delete BigQuery tables and views. This patch
adds a delete operator that allows us to do so.

Closes #1798 from illop/BigQueryDeleteOperator


> Create operator to delete tables in BigQuery
> --------------------------------------------
>
>                 Key: AIRFLOW-509
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-509
>             Project: Apache Airflow
>          Issue Type: New Feature
>            Reporter: Ilya Rakoshes
>            Assignee: Ilya Rakoshes
>             Fix For: Airflow 1.8
>
>
> Airflow currently has several operators for BigQuery, but none of them are able to delete
tables. The request is to create a new operator that will be able to do this.
> An important application of this is to help refresh views for which the schema of the
underlying tables is subject to change. BigQuery has no functionality to refresh the schema
of a view, so if the underlying table changes the schema will not have the new data and things
like auto-completion will not work in the UI. By deleting a view and re-creating it, we can
force a schema refresh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message