thrift-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarry Shaw (JIRA)" <j...@apache.org>
Subject [jira] [Created] (THRIFT-4677) UnicodeDecodeError in Python3
Date Fri, 30 Nov 2018 13:42:00 GMT
Jarry Shaw created THRIFT-4677:
----------------------------------

             Summary: UnicodeDecodeError in Python3
                 Key: THRIFT-4677
                 URL: https://issues.apache.org/jira/browse/THRIFT-4677
             Project: Thrift
          Issue Type: Bug
          Components: Python - Library
         Environment: Operating System: Windows 10 Pro (Simplified Chinese)

Python Interpreter: Python 3.6.6

{{osquery}} Version: 3.3.0

{{osquery-python}} Version: 3.0.5

 
            Reporter: Jarry Shaw
         Attachments: compat.py

This is an issue occurred when using [osquery-python|https://github.com/osquery/osquery-python]
(Python binding of [osquery|https://osquery.io/] by Facebook).

When querying, {{UnicodeDecodeError}} raised with error message: "{{'utf-8' codec can't decode
byte 0xc3 in position 0: invalid continuation byte}}" from {{thrift.compat.binary_to_str}},
which is because the encoding of {{bin_val}} parameter should be "{{gbk}}".

Possible approaches are:
 * add a parameter for user to determine encodings
 * get the system encoding through {{locale.getpreferredencoding()}}
 * call {{bin_val.decode}} with {{errors='replace'}} or {{errors='ignore'}} parameter
 * introduce {{chardet}} to try and resolve encoding problems

The attachment is my hack solution to this issue (through not perfect).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message