hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Release1.0Requirements" by DougCutting
Date Wed, 08 Oct 2008 17:27:44 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by DougCutting:

  ''I wasn't suggesting changing the RPC protoocol but merely the layer above the rpc protocol
- the rpc type system and/or serialization.
  For example, protocol buffers lets you use your own transport. So may thrift. I would also
like to wait till a good rpc solution is available. However I would like to understand what
you think is missing in the current RPC landscape? Are there specific features that you see
missing in some of the rpc solutions or are you waiting for CISCO's Etch solution to make
a decision. Etch looks very attractive on the surface. I am also torn by this decision because
it seems worth waiting till Etch is published before making our decision. It would help if
you could elaborate on your thoughts on the rpc issue. --SanjayRadia''
+ ''What's missing from the current RPC landscape?  Mostly transport-layer stuff.  (1) Transport
versioning.  Thrift doesn't provide transport-level handshakes, so we'd probably need to implement
our own transport.  This is possible, and we'd have to do it for protocol buffers too, but
we might not with Etch.  (2) Async transport.  For performance we need async servers at least,
and probably async clients.  Requests and responses must be multiplexed over shared connections.
 Thrift doesn't yet provide this for Java.  Etch may solve both of these or none or have other
problems.  It would be nice to get as much as possible from an external project, reinventing
the minimum.  So we should certainly start experimenting now.  Someone could, e.g., port Thrift
and/or protocol buffers to run on top of Hadoop's existing transport layer.  We could immediately
incorporate any improvements that make the transport more easily usable for Thrift and Protocol
Buffers, and we'd probably identi
 fy other issues in the process.  Fundamentally, I don't think switching the RPC is a move
we can schedule without more work up front.  But we should certainly start experimenting now.
  ''Language Neutral - yes it will mean duplicated client-side and hence more work. From what
I have observed in the design discussion,  keeping the client side small was a criteria because
we were expecting language neutral protocols down the road. Do you feel that we should not
bother with language neutral protocols at all? --SanjayRadia''

View raw message