libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anthonys...@apache.org
Subject libcloud git commit: PY3: Reduce AWS S3 test errors if lxml is present
Date Sat, 23 Apr 2016 03:11:36 GMT
Repository: libcloud
Updated Branches:
  refs/heads/trunk 77afd30dc -> e6a6676a5


PY3: Reduce AWS S3 test errors if lxml is present

At least lxml should raise ValueError pretty quickly if self.body is a str type, and it was
almost certainly assumed to be 'utf-8', this being the web, so we'll pay the price of the
decode & encode on the production happy path to get a probable overall speedup from lxml.
http://lxml.de/performance.html

Seems cleaner than alternate approaches like trying to inject an additional Response.content
attribute to work with the Python 3 bytes type or changing the response.body to a bytes type
(causes JSON tests to break). e.g. See https://github.com/apache/libcloud/pull/767

I don't have an aliyun.py, etc driver to test against so don't feel comfortable changing that
though happy for others to do so.

In my local Python 3.5+lxml virtualenv, running "python setup.py test":

BEFORE
----------------------------------------------------------------------
Ran 5439 tests in 24.415s

FAILED (failures=4, errors=595, skipped=14)

AFTER
----------------------------------------------------------------------
Ran 5439 tests in 27.036s

FAILED (failures=4, errors=176, skipped=14)

Closes #769


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/e6a6676a
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/e6a6676a
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/e6a6676a

Branch: refs/heads/trunk
Commit: e6a6676a5753569541b914cae614819113a8b042
Parents: 77afd30
Author: Peter Schmidt <peter@peterjs.com>
Authored: Wed Apr 20 16:33:37 2016 +1000
Committer: anthony-shaw <anthonyshaw@apache.org>
Committed: Sat Apr 23 13:11:09 2016 +1000

----------------------------------------------------------------------
 libcloud/common/base.py          | 6 +++++-
 libcloud/test/storage/test_s3.py | 6 +++++-
 2 files changed, 10 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/libcloud/blob/e6a6676a/libcloud/common/base.py
----------------------------------------------------------------------
diff --git a/libcloud/common/base.py b/libcloud/common/base.py
index bda758b..318d28e 100644
--- a/libcloud/common/base.py
+++ b/libcloud/common/base.py
@@ -270,7 +270,11 @@ class XmlResponse(Response):
             return self.body
 
         try:
-            body = ET.XML(self.body)
+            try:
+                body = ET.XML(self.body)
+            except ValueError:
+                # lxml wants a bytes and tests are basically hard-coded to str
+                body = ET.XML(self.body.encode('utf-8'))
         except:
             raise MalformedResponseError('Failed to parse XML',
                                          body=self.body,

http://git-wip-us.apache.org/repos/asf/libcloud/blob/e6a6676a/libcloud/test/storage/test_s3.py
----------------------------------------------------------------------
diff --git a/libcloud/test/storage/test_s3.py b/libcloud/test/storage/test_s3.py
index 2a6e873..af44f6d 100644
--- a/libcloud/test/storage/test_s3.py
+++ b/libcloud/test/storage/test_s3.py
@@ -338,7 +338,11 @@ class S3MockRawResponse(MockRawResponse):
             return self.body
 
         try:
-            body = ET.XML(self.body)
+            try:
+                body = ET.XML(self.body)
+            except ValueError:
+                # lxml wants a bytes and tests are basically hard-coded to str
+                body = ET.XML(self.body.encode('utf-8'))
         except:
             raise MalformedResponseError("Failed to parse XML",
                                          body=self.body,


Mime
View raw message