airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maximebeauche...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results
Date Fri, 01 Jul 2016 16:54:16 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 0a94cae4b -> a9f9fe560


[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results

Dear Airflow Maintainers,

Please accept this PR that addresses the following issues:
- *(replace with a link to AIRFLOW-X)*

Detail:
The function `get_results` implemented in the `HiveServer2Hook` does not execute multiple
commands passed to it in a list, in the singular cursor scope. This has caused SQL statements
that depend on the execution of `add jar` and `set` commands to fail as they are being executed
in different cursor scopes which are not persistent.
The code has been updated to have the cursor object persistent.

Testing Done:

Reminders for contributors:
* Your PR's title must reference an issue on [Airflow's JIRA](https://issues.apache.org/jira/browse/AIRFLOW/).
For example, a PR called "[AIRFLOW-1] My Amazing PR" would close JIRA issue #1. Please open
a new issue if required!
* Please squash your commits when possible and follow the [7 rules of good Git commits](http://chris.beams.io/posts/git-commit/#seven-rules).

mistercrunch

Closes #1629 from sherwian/hivehook_fx


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a9f9fe56
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/a9f9fe56
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/a9f9fe56

Branch: refs/heads/master
Commit: a9f9fe560a7ca7bf03ee577616ab38344e0d73b7
Parents: 0a94cae
Author: sherwian_williamson <sherwian_williamson@airbnb.com>
Authored: Fri Jul 1 09:52:55 2016 -0700
Committer: Maxime Beauchemin <maximebeauchemin@gmail.com>
Committed: Fri Jul 1 09:53:19 2016 -0700

----------------------------------------------------------------------
 airflow/hooks/hive_hooks.py | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/a9f9fe56/airflow/hooks/hive_hooks.py
----------------------------------------------------------------------
diff --git a/airflow/hooks/hive_hooks.py b/airflow/hooks/hive_hooks.py
index e8e10b4..eaad390 100644
--- a/airflow/hooks/hive_hooks.py
+++ b/airflow/hooks/hive_hooks.py
@@ -527,22 +527,22 @@ class HiveServer2Hook(BaseHook):
                 'data': [],
                 'header': [],
             }
+            cur = conn.cursor()
             for statement in hql:
-                with conn.cursor() as cur:
-                    cur.execute(statement)
-                    records = []
-                    try:
-                        # impala Lib raises when no results are returned
-                        # we're silencing here as some statements in the list
-                        # may be `SET` or DDL
-                        records = cur.fetchall()
-                    except ProgrammingError:
-                        logging.debug("get_results returned no records")
-                    if records:
-                        results = {
-                            'data': records,
-                            'header': cur.description,
-                        }
+                cur.execute(statement)
+                records = []
+                try:
+                    # impala Lib raises when no results are returned
+                    # we're silencing here as some statements in the list
+                    # may be `SET` or DDL
+                    records = cur.fetchall()
+                except ProgrammingError:
+                    logging.debug("get_results returned no records")
+                if records:
+                    results = {
+                        'data': records,
+                        'header': cur.description,
+                    }
             return results
 
     def to_csv(


Mime
View raw message