hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: HTTP transport?
Date Tue, 29 Sep 2009 22:11:27 GMT
Doug Cutting wrote:
> stack wrote:
>> What do you think the path on the first line look like? Will it be a 
>> method
>> name or will it be customizable?
> 
> Avro RPC currently includes the message name in the payload, so, unless 
> that changes, for Avro RPC, we'd probably use a different URL per 
> protocol.  As a convention we might use the namespace-qualified protocol 
> name as the URL path.
> 
> Alternately, we could try to make Avro's RPC more HTTP-friendly, and 
> pull stuff out of Avro's payload into HTTP headers.  The downside of 
> that would be that, if we still wish to support non-HTTP transports, 
> we'd end up with duplicated logic.

Keeping Avro payload independent of transport seems pretty useful, at 
least for now. As understand Avro payload is Avro 'proper' (i.e. it 
issupported in all the languages supported by Avro... and other goodies).

I just noticed AVRO-129 and it seems like a great example of using HTTP 
transport.

Does this mean current Avro RPC transport (an improved version of Hadoop 
RPC) can still exist as long as it supported by developers?

Where does security lie : Avro or Transport layer?
If it is part of Avro : transport layer does not matter for security.
If it is part of transport : How does an app get hold of required 
information (e.g. user identity).
May be 'transceiver' can have an interface that can transfer security 
information between transport layer and Avro.

Raghu.

> If we fully embraced HTTP as Avro's primary RPC transport then it might 
> make sense to move the message name to the URL and to use the HTTP 
> return code to determine whether the response is an error or not. Avro's 
> RPC payload also currently includes request and response metadata, which 
> are functionally redundant with HTTP headers.
> 
>> (In hbase, it might be nice to have path be 
>> /tablename/row/family/qualifier etc).
> 
> It sounds like you'd perhaps like to be able to put RPC request 
> parameters into the URL?  I don't see that being done automatically in a 
> general way for arbitrary parameter types without the URLs getting 
> really ugly and adding a lot of complexity.  For this it might be better 
> to write a servlet filter that constructs the appropriate Avro-format 
> request and forwards it to the RPC url.
> 
> Doug


Mime
View raw message