thrift-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aki Sukegawa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (THRIFT-4677) UnicodeDecodeError in Python3
Date Sun, 17 Feb 2019 14:31:00 GMT

    [ https://issues.apache.org/jira/browse/THRIFT-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770412#comment-16770412
] 

Aki Sukegawa commented on THRIFT-4677:
--------------------------------------

AFAIK Apache Thrift's string wire format is tied to UTF8.

> UnicodeDecodeError in Python3
> -----------------------------
>
>                 Key: THRIFT-4677
>                 URL: https://issues.apache.org/jira/browse/THRIFT-4677
>             Project: Thrift
>          Issue Type: Bug
>          Components: Python - Library
>         Environment: Operating System: Windows 10 Pro (Simplified Chinese)
> Python Interpreter: Python 3.6.6
> {{osquery}} Version: 3.3.0
> {{osquery-python}} Version: 3.0.5
>  
>            Reporter: Jarry Shaw
>            Priority: Critical
>         Attachments: compat.py
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> This is an issue occurred when using [osquery-python|https://github.com/osquery/osquery-python]
(Python binding of [osquery|https://osquery.io/] by Facebook).
> When querying, {{UnicodeDecodeError}} raised with error message: "{{'utf-8' codec can't
decode byte 0xc3 in position 0: invalid continuation byte}}" from {{thrift.compat.binary_to_str}},
which is because the encoding of {{bin_val}} parameter should be "{{gbk}}".
> Possible approaches are:
>  * add a parameter for user to determine encodings
>  * get the system encoding through {{locale.getpreferredencoding()}}
>  * call {{bin_val.decode}} with {{errors='replace'}} or {{errors='ignore'}} parameter
>  * introduce {{chardet}} to try and resolve encoding problems
> The attachment is my hack solution to this issue (through not perfect).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message