hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <e...@lifeless.net>
Subject Re: HTTP transport?
Date Mon, 05 Oct 2009 20:43:15 GMT
Doug Cutting wrote:
> More or less.  Except we can probably arrange to omit most of those
> response headers except Content-Length.  Are any others strictly required?

Content-Type and Server are probably unavoidable. Some of the others are
extremely helpful during development / debugging / etc. It depends on
how "open" you are about HTTP being the transport (i.e. do you let
developers augment these headers to support additional features, etc.).
This may not make sense in the context of something specialized like
Avro transport.

> I today implemented a simple HTTP-based transport for Avro:
> 
>   https://issues.apache.org/jira/browse/AVRO-129
> 
> In some simple benchmarks I am able to make over 5000 sequential
> RPCs/second, each with ~100 bytes of response payload.

Just out of curiousity, were you using HTTP keep alive? During testing
on a project a few years ago, I found a huge difference if Keep Alive is
supported. In retrospect, that should have been obvious. I'd imagine the
usage pattern here would be a large number of repeated calls between the
same client / server within a short period of time; perfect for KA.

Regards.
-- 
Eric Sammer
eric@lifless.net
http://esammer.blogspot.com

Mime
View raw message