thrift-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarry Shaw (JIRA)" <>
Subject [jira] [Created] (THRIFT-4677) UnicodeDecodeError in Python3
Date Fri, 30 Nov 2018 13:42:00 GMT
Jarry Shaw created THRIFT-4677:

             Summary: UnicodeDecodeError in Python3
                 Key: THRIFT-4677
             Project: Thrift
          Issue Type: Bug
          Components: Python - Library
         Environment: Operating System: Windows 10 Pro (Simplified Chinese)

Python Interpreter: Python 3.6.6

{{osquery}} Version: 3.3.0

{{osquery-python}} Version: 3.0.5

            Reporter: Jarry Shaw

This is an issue occurred when using [osquery-python|]
(Python binding of [osquery|] by Facebook).

When querying, {{UnicodeDecodeError}} raised with error message: "{{'utf-8' codec can't decode
byte 0xc3 in position 0: invalid continuation byte}}" from {{thrift.compat.binary_to_str}},
which is because the encoding of {{bin_val}} parameter should be "{{gbk}}".

Possible approaches are:
 * add a parameter for user to determine encodings
 * get the system encoding through {{locale.getpreferredencoding()}}
 * call {{bin_val.decode}} with {{errors='replace'}} or {{errors='ignore'}} parameter
 * introduce {{chardet}} to try and resolve encoding problems

The attachment is my hack solution to this issue (through not perfect).

This message was sent by Atlassian JIRA

View raw message