cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Bannister (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-14320) dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes
Date Sun, 18 Mar 2018 01:07:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403822#comment-16403822
] 

Patrick Bannister commented on CASSANDRA-14320:
-----------------------------------------------

I checked  cqlsh_tests/cqlsh_copy_tests.py and offline_tools_test.py. Neither is impacted
by this issue, since ccm was recently changed to decode stdout and stderr from subprocess
to a string before returning it.

> dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes
> -------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-14320
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Testing
>            Reporter: Patrick Bannister
>            Priority: Minor
>              Labels: Python3, dtest, python3
>             Fix For: 3.0.x, 3.11.x
>
>
> JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its _query function
tries to use json.loads (which only accepts string input) on a bytes object.
> {code:java}
>     def _query(self, body, verbose=True):
>         request_data = json.dumps(body).encode("utf-8")
>         url = 'http://%s:8778/jolokia/' % (self.node.network_interfaces['binary'][0],)
>         req = urllib.request.Request(url)
>         response = urllib.request.urlopen(req, data=request_data, timeout=10.0)
>         if response.code != 200:
>             raise Exception("Failed to query Jolokia agent; HTTP response
code: %d; response: %s" % (response.code, response.readlines()))
>         raw_response = response.readline() # response is http.client.HTTPResponse,
which subclasses RawIOBase, which returns bytes when read
>         response = json.loads(raw_response) # this raises a TypeError now
>         if response['status'] != 200:
>             stacktrace = response.get('stacktrace')
>             if stacktrace and verbose:
>                 print("Stacktrace from Jolokia error follows:")
>                 for line in stacktrace.splitlines():
>                     print(line)
>             raise Exception("Jolokia agent returned non-200 status: %s" %
(response,))
>         return response{code}
> This can be seen clearly by running the deprecated repair tests (repair_tests/deprecated_repair_test.py).
They all fail right now because of this TypeError.
> This is a side effect of the migration to Python 3, which makes bytes objects fundamentally
different from strings. This will also happen anytime we try to json.loads data returned from
stdout or stderr piped from subprocess. I need to take a closer look at offline_tools_test.py
and cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
> We can fix this issue by decoding bytes objects to strings before calling json.loads().
For example, in the above:
> {code:java}
>         response = json.loads(raw_response.decode(encoding='utf-8')){code}
> I have a fix for the JolokiaAgent problem - I'll submit a pull request to cassandra-dtest
once I have this issue number to reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message