thrift-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Slee (JIRA)" <j...@apache.org>
Subject [jira] Commented: (THRIFT-155) Python THttpClient should handle opening HTTP layer on each request automatically
Date Tue, 07 Oct 2008 01:29:44 GMT

    [ https://issues.apache.org/jira/browse/THRIFT-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637346#action_12637346
] 

Mark Slee commented on THRIFT-155:
----------------------------------

Yeah, that's what I was thinking about. I wasn't worried about the isOpen() overhead, rather
the fact that it results in a close() call, which we might really not want with a better underlying
HTTP implementation. It's annoying that there doesn't seem to be a built-in keep-alive HTTP
client implementation in the Python libraries yet. I'd rather avoid pulling in more 3rd party
python libs for this, and I'd also rather not re-implement HTTP in Thrift. Do you know if
httplib2 is vying to make its way into the standard Python libs?

The other thing that really irks me in Python networking is the lack of independent socket
timeout controls. With most of the HTTP/url libraries, it's impossible to set connection timeouts
without modifying socket.timeout directly  -- which feels like very dangerous practice in
a large project that might have many dependencies on socket -- or breaking into the abstraction's
internals to setsockopt on the underlying object, also not pretty. I'm glad that httplib2
has added proper timeout support.

> Python THttpClient should handle opening HTTP layer on each request automatically
> ---------------------------------------------------------------------------------
>
>                 Key: THRIFT-155
>                 URL: https://issues.apache.org/jira/browse/THRIFT-155
>             Project: Thrift
>          Issue Type: Improvement
>          Components: Library (Python)
>            Reporter: Dave Engberg
>         Attachments: THttpClient.py.diff
>
>
> The HTTP transport for other languages (like Java) will automatically open the HTTP connection
on each Thrift message in the flush() method.
> The Python THttpClient requires you to explicitly call .open() on the low-level transport
before every Thrift invocation.  If you don't do this, you'll get a "ResponseNotReady" error
out of httplib:
> ...
>     self._oprot.trans.flush()
>   File "/Users/dave/dev/work/en-thrift/target/evernote-api-1.10/lib/python/thrift/transport/THttpClient.py",
line 60, in flush
>     self.code, self.message, self.headers = self.__http.getreply()
>   File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/httplib.py",
line 1195, in getreply
>     response = self._conn.getresponse()
>   File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/httplib.py",
line 914, in getresponse
>     raise ResponseNotReady()
> httplib.ResponseNotReady
> I'm submitting a small patch to THttpClient.py that will automatically handle the opening
of the required low-level HTTP connection with each Thrift message in the beginning of the
flush() method:
> Index: lib/py/src/transport/THttpClient.py
> ===================================================================
> --- lib/py/src/transport/THttpClient.py	(revision 701711)
> +++ lib/py/src/transport/THttpClient.py	(working copy)
> @@ -37,6 +37,10 @@
>      self.__wbuf.write(buf)
>  
>    def flush(self):
> +    if self.isOpen():
> +      self.close()
> +    self.open();
> +
>      # Pull data out of buffer
>      data = self.__wbuf.getvalue()
>      self.__wbuf = StringIO()

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message