hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: [PROPOSAL] new subproject: Avro
Date Mon, 06 Apr 2009 20:48:38 GMT
George Porter wrote:
> One thing that you might consider is adding the ability to add path 
> state to the serialization to better support tracing and instrumentation 
> of the RPC layer with Avro.

Such cross-system tracing would be great, but I think this might best be 
done as part of request and response metadata, rather than made part of 
the user's request and response data.

You and I worked this nearly through for Hadoop's existing RPC:

http://issues.apache.org/jira/browse/HADOOP-4049

Unfortunately, that patch stalled awaiting benchmarks, and now is stale. 
  Avro currently specifies a request and response payload format, but 
does not say much about metadata.  In an HTTP-based transport, tracing 
might be done through headers.  As we adapt Hadoop's optimized RPC 
transport to support Avro messages we should probably insert a generic 
metadata layer, analogous to HTTP's headers, to support features such as 
this.

Doug

Mime
View raw message