thrift-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James E. King III (JIRA)" <>
Subject [jira] [Closed] (THRIFT-146) Make Thrift EventMachine Compatible
Date Thu, 03 Jan 2019 03:37:00 GMT


James E. King III closed THRIFT-146.
    Resolution: Incomplete
      Assignee: James E. King III

Just doing some backlog cleaning and found this, it was marked with a fix version of 1.0 which
I removed.  It's been open for a good 6 years without any action so I am going to close it
as incomplete.  Given the age it's quite possible there are better ways to do this in Ruby
these days anyway.

> Make Thrift EventMachine Compatible
> -----------------------------------
>                 Key: THRIFT-146
>                 URL:
>             Project: Thrift
>          Issue Type: Improvement
>          Components: Ruby - Library
>         Environment: Ruby and EventMachine
>            Reporter: Ben Taitelbaum
>            Assignee: James E. King III
>            Priority: Trivial
>         Attachments: 0001-Ruby-client-increments-sequence-ids.patch, 0002-Added-rb-eventmachine-option-to-the-compiler.patch,
0003-Added-ruby-eventmachine-compatible-client-code.patch, 0004-Added-EventMachineTransport-class.patch,
0005-rb-eventmachine-can-have-a-separate-namespace-than-r.patch, 0006-Added-eventmachine-as-a-dev-dependency.patch,
0007-Generate-eventmachine-bindings-for-tests.patch, 0008-Initial-event-machine-test.patch,
0009-Added-an-asnyc-test-for-EventMachine.patch, 0010-Added-test-for-oneway-functions-with-eventmachine.patch,
0011-Added-eventmachine-test-for-void-functions.patch, 0012-Eventmachine-triggers-errback-on-connect-errors.patch,
0013-Added-timeout-option-to-eventmachine-transport.patch, 0014-Eventmachine-spec-cleanup-for-ruby-1.8.patch,
0015-Added-EventMachine-tutorial-example.patch, 0016-EM-Trigger-error-callbacks-on-connection-close.patch,
> I wrote a prototype EventMachine client/server for Thrift (BinaryProtocol), and while
it's not fully functional, the results are so promising, that I think it would be worth it
to either include an EM client/server with the thrift distribution, or at least rework the
libraries so that they can support EM without having to copy/paste all the code I did.
> I'm attaching my prototype as an example. It requires the eventmachine and statemachine
gems. It's wired to support a simple echo service. It's a little confusing in that I reused
the same statemachine for both client and server, doesn't handle exceptions, and has a bug
where it'll handle all requests from one client before handling requests from another, but
hey, it's a prototype.
> EM uses very different semantics from typical client/server, and this presents difficulty
wiring it to the current thrift protocols. For example, while currently, you can call call to read n bytes off of a stream, or wait until they're available, with EM, if
n bytes aren't available, you simply don't handle the call yet. I handled this by trying to
read as many bytes as need be, and keeping track of how much to backup in the stream, ultimately
only transitioning to the next state in the statemachine if we were able to read all the bytes
needed. This actually works fairly well, but it would be better if we had a byte count at
each step so we could tell right away if we had enough data.
> The use of EM allows for new asynchronous semantics for thrift calls, in that the client
returns right away from a call, and its callback is called when a result (or exception) is
available. Backwards compatibility could still be achieved by blocking on a call, but at least
in my experience, in most of the cases having callbacks would be a huge optimization. There
are some cases where I do want to either ensure that calls happen sequentially, or are at
least handled in the same order.
> These are the changes I believe would have to be made in order to support an EM thrift
> 1. syntax to indicate that certain methods can be called asynchronously with callbacks.
Since async is already a reserved keyword, and is changing to noreturn, what about using this?
> 2. I would propose moving all reading and writing to the same class. So instead of having
ThriftStruct#read, we would have Protocol#read_struct. It seems out of place to me to have
a model handle its own reading / writing.
> 3. support for byte counts in the protocols. It would be helpful to have a clear way
to get the number of bytes that must be read to retrieve message headers, arguments, and results.
Alternatively, the protocols could support read_or_backup semantics, but as you can see in
my prototype, this can get messy.
> Please let me know what you think about this proposal, and the possibility of using EventMachine.

This message was sent by Atlassian JIRA

View raw message