avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell" <pwend...@cloudera.com>
Subject Re: Review Request: AVRO-595.patch.v2.txt
Date Tue, 03 Aug 2010 22:21:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/390/
-----------------------------------------------------------

(Updated 2010-08-03 15:21:00.720884)


Review request for Avro.


Changes
-------

Hey Phil. Turns out I was a little behind trunk, and an API change on HTTPServer was causing
the bug. I also added a small bug fix that is required for this patch (3 lines in Requestor.java)
and added better tearing-down for the unit tests. Finally, there are some minor changes to
make checkstyle happy.


Summary
-------

Here is a patch update based on the preliminary feedback. I dropped
the querying stuff since it's not yet used. This patch adds the logic
behind aggregating partial spans (which will have been collected from
several remote hosts) and creating traces (extracting RPC call trees
from a set of spans).

We talked off-line about redundancy in the data that is collected. We
could get away with only recording a spanID and a parentSpanID for
each span, then we can back-out each trace by following these data (we
get to the root of a trace when we see a null parentSpanID). By adding
a traceID unique to each call tree, it is easier to partition the work
of collecting spans into traces as we can easily group spans by trace
before we try back-out a tree (perhaps this is why Dapper paper
describes this method). It also lowers the chance of ID collision
since spanID and parentSpanID are specific to one traceID.


Diffs (updated)
-----

  lang/java/src/java/org/apache/avro/ipc/Requestor.java 2661aaa 
  lang/java/src/java/org/apache/avro/ipc/trace/InMemorySpanStorage.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/SpanAggregator.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/SpanStorage.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/Trace.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/TraceClientServlet.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/TraceNode.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/TracePlugin.java PRE-CREATION 
  lang/java/src/java/org/apache/avro/ipc/trace/TracePluginConfiguration.java PRE-CREATION

  lang/java/src/java/org/apache/avro/ipc/trace/Util.java PRE-CREATION 
  lang/java/src/test/java/org/apache/avro/ipc/trace/TestBasicTracing.java PRE-CREATION 
  lang/java/src/test/java/org/apache/avro/ipc/trace/TestEndToEndTracing.java PRE-CREATION

  lang/java/src/test/java/org/apache/avro/ipc/trace/TestSpanAggregation.java PRE-CREATION

  lang/java/src/test/java/org/apache/avro/ipc/trace/TestSpanTraceFormation.java PRE-CREATION

  share/schemas/org/apache/avro/ipc/trace/avroTrace.avdl PRE-CREATION 
  share/schemas/org/apache/avro/ipc/trace/avroTrace.avpr PRE-CREATION 

Diff: http://review.cloudera.org/r/390/diff


Testing
-------

Several unit-tests are present in this review. See testTraceAndCollection() in TestEndToEndTracing.java
for a clear example of the creation and subsequent aggregation of trace data.


Thanks,

Patrick


Mime
View raw message