hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Radia <sra...@yahoo-inc.com>
Subject Re: HTTP transport?
Date Tue, 29 Sep 2009 18:52:09 GMT

On Sep 28, 2009, at 3:42 PM, Doug Cutting wrote:

> Owen O'Malley wrote:
> > I've got concerns about this. Both tactical and strategic. The  
> tactical
> > problem is that I need to get security (both Kerberos and token)  
> in to
> > 0.22. I'd really like to get Avro RPC into 0.22. I'd like both to be
> > done roughly in 5 months. If you switch off of the current RPC  
> code base
> > to a completely new RPC code base, I don't see that happening.
> What transport do you expect to use with Avro?  If wire- 
> compatibility is
> a goal, and that includes access from languages besides Java, then we
> must use a transport that's well-specified and Java-independent.  HTTP
> is both of these.  The existing Hadoop RPC protocol is not.
> We could adapting Hadoop's existing RPC transport to be well-specified
> and language independent.  This is perhaps not a huge task, but it  
> feels
> to me a bit like re-inventing much of what's already in HTTP clients  
> and
> servers these days: connection-pooling, async servers, etc.

Wrt  connection pooling/async servers: Can't we use the same libraries  
that Jetty and Tomcat use?

> grate Kerberos with
> Jetty than with a homegrown protocol and server?
> >   - very expensive on the wire encryption (ssl)
> If we don't use HTTP, will we be providing on-wire encryption?  If  
> not,
> this is moot.

Yes we are expecting to use encryption down the road.
> Finally, need to have secure HTTP-based access anyway, right?  If we  
> use
> HTTP as our RPC transport mightn't we reuse most of that effort?
> Doug

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message