superset-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maximebeauche...@apache.org
Subject [incubator-superset] branch master updated: [sql lab] Fix issue around VARBINARY type in Presto (#5121)
Date Wed, 20 Jun 2018 19:17:02 GMT
This is an automated email from the ASF dual-hosted git repository.

maximebeauchemin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
     new 409ac68  [sql lab] Fix issue around VARBINARY type in Presto (#5121)
409ac68 is described below

commit 409ac6824add4f5d9ec868e8fb8b2506d4602dfd
Author: Maxime Beauchemin <maximebeauchemin@gmail.com>
AuthorDate: Wed Jun 20 12:16:59 2018 -0700

    [sql lab] Fix issue around VARBINARY type in Presto (#5121)
    
    When receiving a VARBINARY field out of Presto, it shows up as type
    `bytes` out of the pyhive driver. Then the pre 3.15 version of
    simplejson attempts to convert it to utf8 by default and it craps out.
    
    I bumped to simplejson>=3.25.0 and set `encoding=None` as documented
    here
    https://simplejson.readthedocs.io/en/latest/#basic-usage so that we can
    handle bytes on our own.
---
 requirements.txt       | 2 +-
 setup.py               | 2 +-
 superset/utils.py      | 6 +++++-
 superset/views/core.py | 6 +++++-
 4 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/requirements.txt b/requirements.txt
index 76952f2..a0fd867 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -29,7 +29,7 @@ python-dateutil==2.6.1
 python-geohash==0.8.5
 pyyaml==3.12
 requests==2.18.4
-simplejson==3.13.2
+simplejson==3.15.0
 six==1.11.0
 sqlalchemy==1.2.2
 sqlalchemy-utils==0.32.21
diff --git a/setup.py b/setup.py
index c1f555f..7adccb7 100644
--- a/setup.py
+++ b/setup.py
@@ -85,7 +85,7 @@ setup(
         'python-geohash',
         'pyyaml>=3.11',
         'requests',
-        'simplejson',
+        'simplejson>=3.15.0',
         'six',
         'sqlalchemy',
         'sqlalchemy-utils',
diff --git a/superset/utils.py b/superset/utils.py
index eec2530..61d5cb6 100644
--- a/superset/utils.py
+++ b/superset/utils.py
@@ -310,7 +310,6 @@ def datetime_f(dttm):
 
 
 def base_json_conv(obj):
-
     if isinstance(obj, numpy.int64):
         return int(obj)
     elif isinstance(obj, numpy.bool_):
@@ -323,6 +322,11 @@ def base_json_conv(obj):
         return str(obj)
     elif isinstance(obj, timedelta):
         return str(obj)
+    elif isinstance(obj, bytes):
+        try:
+            return '{}'.format(obj)
+        except Exception:
+            return '[bytes]'
 
 
 def json_iso_dttm_ser(obj, pessimistic=False):
diff --git a/superset/views/core.py b/superset/views/core.py
index 61fe7eb..3a0f28f 100755
--- a/superset/views/core.py
+++ b/superset/views/core.py
@@ -2464,7 +2464,11 @@ class Superset(BaseSupersetView):
                     rendered_query,
                     return_results=True)
             payload = json.dumps(
-                data, default=utils.pessimistic_json_iso_dttm_ser, ignore_nan=True)
+                data,
+                default=utils.pessimistic_json_iso_dttm_ser,
+                ignore_nan=True,
+                encoding=None,
+            )
         except Exception as e:
             logging.exception(e)
             return json_error_response('{}'.format(e))


Mime
View raw message