airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bo...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-2146] Resolve issues with BQ using DbApiHook methods
Date Sun, 25 Feb 2018 11:16:37 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master d7b5d09d4 -> f2d10ef7b


[AIRFLOW-2146] Resolve issues with BQ using DbApiHook methods

- Resolves issues with using methods like
`get_records()` from `BigQueryHook` which is
extended from `DbApiHook`.
- Reverting one changed file from
https://github.com/apache/incubator-
airflow/pull/3031 to use BigQuery Hook again
instead of `GoogleCloudBaseHook`
- Fix `conn_type` of `bigquery_default` connection

Closes #3073 from kaxil/AIRFLOW-2146


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/f2d10ef7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/f2d10ef7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/f2d10ef7

Branch: refs/heads/master
Commit: f2d10ef7b465080584fe96590775b6a2696a38ef
Parents: d7b5d09
Author: Kaxil Naik <kaxilnaik@gmail.com>
Authored: Sun Feb 25 12:16:28 2018 +0100
Committer: Bolke de Bruin <bolke@xs4all.nl>
Committed: Sun Feb 25 12:16:28 2018 +0100

----------------------------------------------------------------------
 airflow/models.py   | 4 ++--
 airflow/utils/db.py | 3 ++-
 tests/core.py       | 1 -
 3 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/f2d10ef7/airflow/models.py
----------------------------------------------------------------------
diff --git a/airflow/models.py b/airflow/models.py
index b318ea6..f9f3fbc 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -662,8 +662,8 @@ class Connection(Base, LoggingMixin):
                 from airflow.hooks.mysql_hook import MySqlHook
                 return MySqlHook(mysql_conn_id=self.conn_id)
             elif self.conn_type == 'google_cloud_platform':
-                from airflow.contrib.hooks.gcp_api_base_hook import GoogleCloudBaseHook
-                return GoogleCloudBaseHook(gcp_conn_id=self.conn_id)
+                from airflow.contrib.hooks.bigquery_hook import BigQueryHook
+                return BigQueryHook(bigquery_conn_id=self.conn_id)
             elif self.conn_type == 'postgres':
                 from airflow.hooks.postgres_hook import PostgresHook
                 return PostgresHook(postgres_conn_id=self.conn_id)

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/f2d10ef7/airflow/utils/db.py
----------------------------------------------------------------------
diff --git a/airflow/utils/db.py b/airflow/utils/db.py
index 64ce220..2a38424 100644
--- a/airflow/utils/db.py
+++ b/airflow/utils/db.py
@@ -103,7 +103,8 @@ def initdb():
             schema='default'))
     merge_conn(
         models.Connection(
-            conn_id='bigquery_default', conn_type='bigquery'))
+            conn_id='bigquery_default', conn_type='google_cloud_platform',
+            schema='default'))
     merge_conn(
         models.Connection(
             conn_id='local_mysql', conn_type='mysql',

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/f2d10ef7/tests/core.py
----------------------------------------------------------------------
diff --git a/tests/core.py b/tests/core.py
index f76f65e..45a6e34 100644
--- a/tests/core.py
+++ b/tests/core.py
@@ -1039,7 +1039,6 @@ class CliTests(unittest.TestCase):
         # expected:
         self.assertIn(['aws_default', 'aws'], conns)
         self.assertIn(['beeline_default', 'beeline'], conns)
-        self.assertIn(['bigquery_default', 'bigquery'], conns)
         self.assertIn(['emr_default', 'emr'], conns)
         self.assertIn(['mssql_default', 'mssql'], conns)
         self.assertIn(['mysql_default', 'mysql'], conns)


Mime
View raw message