hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <e...@lifeless.net>
Subject Re: HTTP transport?
Date Mon, 05 Oct 2009 20:53:34 GMT

Certainly keep alive will help in this case, if that's what you're
referring to. The server holds the socket for N seconds or M requests,
which ever comes first. What you're saving with KA is the connection
setup / tear down. If you have a lot of cases where the client makes a
single request and goes away, then KA hurts because the server holds the
connection for the KA timeout (N seconds). This *really* helps if you're
using TLS due to the additional connection setup overhead.

It's my opinion and experience that KA helps greatly in the case of many
exchanges between a small to medium number of clients and a server such
as RPC. The anti-example is an ad server or web beacon server, for instance.


Ryan Rawson wrote:
> I have a question about these headers... will they impact the ability to do
> many, but small, rpcs? Imagine you'd need to support 5,000 to 50,000
> rpcs/second. Would this help or hinder?
> On Oct 5, 2009 4:44 PM, "Eric Sammer" <eric@lifeless.net> wrote:
> Doug Cutting wrote: > More or less. Except we can probably arrange to omit
> most of those > response...
> Content-Type and Server are probably unavoidable. Some of the others are
> extremely helpful during development / debugging / etc. It depends on
> how "open" you are about HTTP being the transport (i.e. do you let
> developers augment these headers to support additional features, etc.).
> This may not make sense in the context of something specialized like
> Avro transport.
>> I today implemented a simple HTTP-based transport for Avro: > >
> https://issues.apache.org/jira...
> Just out of curiousity, were you using HTTP keep alive? During testing
> on a project a few years ago, I found a huge difference if Keep Alive is
> supported. In retrospect, that should have been obvious. I'd imagine the
> usage pattern here would be a large number of repeated calls between the
> same client / server within a short period of time; perfect for KA.
> Regards.
> --
> Eric Sammer
> eric@lifless.net
> http://esammer.blogspot.com

Eric Sammer

View raw message