hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: HTTP transport?
Date Tue, 06 Oct 2009 03:30:50 GMT

>> With respect to Avro/Hadoop, I suspect requests from clients to be time
>> clustered.
> That was my thought as well. The thing that gets me is that in the case
> of Hadoop (and the related subprojects) the clients utilizing this
> particular HTTP connection are probably going to be pretty small (maybe
> low thousands?). This is even better for keep alive as there's a solid
> chance you're going to have a high reuse rate. Of course, I'm assuming
> we're talking about things like name node to data node, hbase client to
> region servers, and those types of communications. Even if you just used
> Jetty (or any other thin HTTP 1.1 container that supports KA), one
> should easily be able to see good performance.

Additionally, I realized one more thing -- when a client knows they aren't
likely to send another request soon, they can send a request with
Connection: close.

Well behaved clients can help maximize the benefit.

> Regards.
> --
> Eric Sammer
> eric@lifless.net
> http://esammer.blogspot.com

View raw message