incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: dapper-style tracing in cassandra
Date Tue, 14 Jun 2011 21:02:55 GMT
Sounds a lot like
https://issues.apache.org/jira/browse/CASSANDRA-1123. The main change
you'd want is to allow passing an external trace ID.

On Tue, Jun 14, 2011 at 3:57 PM, Ryan King <ryan@twitter.com> wrote:
> I'll open a ticket on this soon, but I'd like to start a discussion first.
>
> We're working on a distributed tracing system, whose design is
> somewhat inspired by the Google Dapper paper [1]. We have instrumented
> a bunch of our internal services through our custom networking stack
> [2].
>
> In a nutshell, the way it works is that each request is given a trace
> id which gets passed through to each service involved in servicing
> that request. Each hop in that tree is given a span id. Each node logs
> its data to a local agent (we use scribe for this). An aggregator can
> pull the pieces back together so you can do analysis.
>
> I'd like to add the ability to plug tracers into cassandra. Like with
> many things in cassandra, I think like many parts of Cassandra we
> should make this an extensible point with a good default
> implementation in place.
>
> Here's what I propose:
>
> 1. Update the thrift server to allow clients to pass in tracing
> details. I'll have docs soon on how we're doing this internally.
> 2. Add the necessary metadata to each message passed between cassandra
> nodes. This should be easy to Message.java and thread through to the
> places we need it.
> 3. Implement a universally useful version of this– one that's not
> dependent on our system since it may not ever get open-sourced.
> Perhaps writing to local files?
>
> Thoughts? Opinions?
>
> -ryan
>
> 1. http://research.google.com/pubs/pub36356.html
> 2. https://github.com/twitter/finagle/tree/master/finagle-b3
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Mime
View raw message