hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: HTTP transport?
Date Mon, 05 Oct 2009 23:48:24 GMT
Sanjay Radia wrote:
> What about out of order exchange. Will we be able to support that with 
> http transport?

Out-of-order exchange was originally added to Hadoop's RPC when it was a 
part of Nutch.  It's an important optimization for distributed search, 
but it's not clear how important it is currently to Hadoop.

That said, the simple way to deal with this in HTTP is to use a client 
library that pools connections, so that, if a second request to the same 
service is made by another thread in the same client process before the 
first has returned, a second connection is opened.  If this is common, 
the high-water mark of connections on the server will be higher. 
However with an async-io-based server, the number of connections should 
not be a primary bottleneck.  And again, we don't know how common this is.

Doug

Mime
View raw message