zipkin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From adrianc...@apache.org
Subject [incubator-zipkin-b3-propagation] 01/01: Inlines resources hosted on ASF wiki and reverts disclaimers
Date Sun, 16 Jun 2019 06:10:32 GMT
This is an automated email from the ASF dual-hosted git repository.

adriancole pushed a commit to branch revert-asf
in repository https://gitbox.apache.org/repos/asf/incubator-zipkin-b3-propagation.git

commit f1ae2ea0cdceebebfd178b113be574498314e8a8
Author: Adrian Cole <acole@pivotal.io>
AuthorDate: Sun Jun 16 14:10:21 2019 +0800

    Inlines resources hosted on ASF wiki and reverts disclaimers
---
 DISCLAIMER   |   5 --
 NOTICE       |   5 --
 RATIONALE.md | 170 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 README.md    |   2 +-
 STATUS.md    |  44 ++++++++++++++++
 5 files changed, 215 insertions(+), 11 deletions(-)

diff --git a/DISCLAIMER b/DISCLAIMER
deleted file mode 100644
index 728b3bf..0000000
--- a/DISCLAIMER
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache Zipkin B3 Propagation (incubating) is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-Incubation is required of all newly accepted projects until a further review indicates 
-that the infrastructure, communications, and decision making process have stabilized in a
manner consistent with other successful ASF projects. 
-While incubation status is not necessarily a reflection of the completeness or stability
of the code, 
-it does indicate that the project has yet to be fully endorsed by the ASF.
diff --git a/NOTICE b/NOTICE
deleted file mode 100644
index 0041e35..0000000
--- a/NOTICE
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache Zipkin B3 Propagation (incubating)
-Copyright 2019 The Apache Software Foundation
-
-This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
diff --git a/RATIONALE.md b/RATIONALE.md
new file mode 100644
index 0000000..9e1163f
--- /dev/null
+++ b/RATIONALE.md
@@ -0,0 +1,170 @@
+# b3 single header format
+
+In designing the Trace Context format, we made a section called tracestate which holds the
authoritative propagation data.
+
+This section defines a value that could be used as a separate "b3" header, and would be the
same value used in the w3c tracestate field. Specific to the w3c format, this holds data not
in the "traceparent" format, such as parent ID and the debug flag. It would be a completely
non-lossy way to allocate our current headers into one value.
+
+In simplest terms it is a mapping:
+
+```
+b3={x-b3-traceid}-{x-b3-spanid}-{if x-b3-flags 'd' else x-b3-sampled}-{x-b3-parentspanid},
where the last two fields are optional.
+```
+
+For example, the following headers:
+```
+X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
+X-B3-ParentSpanId: 05e3ac9a4f6e3b90
+X-B3-SpanId: e457b5a2e4d86bd1
+X-B3-Sampled: 1
+```
+
+Become one header or state field. For example, if a header:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
+```
+
+Or if using w3c trace context format
+```
+tracestate: b3=80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
+```
+
+Here are some more examples:
+
+A sampled root span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1
+```
+
+A not yet sampled root span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1
+```
+
+A debug RPC child span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-d-05e3ac9a4f6e3b90
+```
+
+Like normal B3, it is valid to omit trace identifiers in order to only propagate a sampling
decision. For example, the following are valid downstream hints:
+
+* don't sample - b3: 0
+* sampled - b3: 1
+* debug - b3: d
+
+*NOTE* this does not match the prefix of traceparent, so we must define ours independently
and consider that the w3c may change in different ways. This is ok as the "tracestate" entries
in w3c format are required to be treated opaque. In other words we can be different on purpose
or by accident of drift in their spec.
+
+## On positional encoding vs nested key/values
+
+Positional encoding is more space efficient and less complicated to parse vs key/value encoding.
For example, the AWS format code in brave is complex due to splitting, dealing with white
space etc. Positional is simple to parse and straight-forward to map. Rationale is same as
w3c traceparent for the most part.
+
+Different than new specs, we expect no additional fields. B3 is a very stable spec and we
are not defining anything new except how to encode it. For this reason, positional should
be fine.
+
+## On putting mandatory fields up front
+
+The trace ID and span ID fields are the only mandatory fields. This would allow fixed-length
parsing for those just correlating on these values. Usually parentid is not used for correlation,
rather scraping. Moreover, this is easier for existing proxies who only create trace identifiers.
+
+Ex: you can control trace identifiers without making a sampling decision like so:
+```
+# root trace and span IDs
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1
+```
+
+## On sampled before parent span ID
+
+When name-values aren't used, it could be confusing which of the equal length fields are
the parent. By placing the sampled flag in-between, we make this more clear. Also, it matches
the prefix of the current [traceparent encoding](https://github.com/w3c/distributed-tracing/blob/master/trace_context/HTTP_HEADER_FORMAT.md#traceparent-field).
+
+## Encoding "not yet sampled"
+
+Leaving out the single-character sampled field is how we encoded the "no decision" state.
This matches the way we used to address this (by leaving out `X-B3-Sampled`).
+
+## Encoding debug
+
+We encode the debug flag (previously `X-B3-Flags: 1`), as the letter 'd' in the same place
as sampled ('1'). This is because debug is a boosted sampled signal. Most implementations
record it out-of-band as `Span.debug=true` to ensure it reaches the collector tier.
+
+```
+# force trace on a root span
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-d
+```
+
+One alternative considered was adding another field just to hold debug (ex a trailing -1).
Not only was this less intuitive, it made parsing harder especially as parentId is also optional.
This was reverted in openzipkin/brave#773
+
+## W3C drift alert
+
+While we should watch out for changes in the [TraceContext spec](https://github.com/w3c/distributed-tracing/issues/8).
For example, if they add a "priority flag", we should keep our impl independent. B3 fields
haven't changed in years and we can lock in something far safer knowing that.
+
+## Why also define as a separate header
+
+We have had continual problems with b3 with technology like JMS. In addition to declaring
this format for w3c Trace Context, we could use it right away as the header "b3". This would
solve all the problems we have like JMS hating hyphens in names, and allow those who opt into
it a consistent format for when they transition to w3c.
+
+openzipkin/brave#584
+
+In other words, in messaging propagation and even normal http, some libraries could choose
to read the "b3" header for the exact same format instead of "X-B3-X"
+
+## Should we use flags instead of two fields for sampled and debug?
+
+We could encode the three sampled states and debug as a single 8-bit field encoded as hex.
If we used flags, an example sampled span would be:
+
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-3-05e3ac9a4f6e3b90
+```
+
+Notice this is 3, not 1. That's because if using bit field we need to tell the difference
between unsampled and no sampling decision. This could be confusing to people.
+
+On flags, in java, we already encode sampled and debug state internally flags like this internally
+```java
+  static final int FLAG_SAMPLED = 1 << 1;
+  static final int FLAG_SAMPLED_SET = 1 << 2;
+  static final int FLAG_DEBUG = 1 << 3;
+
+  static Boolean sampled(int flags) {
+    return (flags & FLAG_SAMPLED_SET) == FLAG_SAMPLED_SET
+        ? (flags & FLAG_SAMPLED) == FLAG_SAMPLED
+        : null;
+  }
+
+  static boolean debug(int flags) {
+    return (flags & FLAG_DEBUG) == FLAG_DEBUG;
+  }
+}
+```
+
+This might be better off than having two fields, although it is less simple as people often
[make mistakes coding bit fields](https://github.com/w3c/distributed-tracing/pull/116), and
X-B3-Flags caused confusion many times here including [#20](https://github.com/openzipkin/b3-propagation/issues/20).
+
+## What about finagle's flags?
+
+If we used flags, we could also do it the same way as [finagle does](https://github.com/twitter/finagle/blob/develop/finagle-core/src/main/scala/com/twitter/finagle/tracing/Flags.scala),
except I think it would be confusing as the length they allocated (64 bits or 16 characters
in hex) was never used in practice.
+
+In practice we could use a single hex character to encode all the flags in our format (that
supports 8 flags). Also, using Finagle's flag encoding could further confusion about the "X-B3-Flags"
header, which in http encoding never has a value besides "1" [#20](https://github.com/openzipkin/b3-propagation/issues/20).
At any rate, we can consider using the first 8 bits of their format as prior art regardless
of if we use it.
+
+```scala
+/*
+ * The debug flag is used to ensure this the current trace passes
+ * all of the sampling stages.
+ */
+  val Debug = 1L << 0 // 1
+
+/**
+ * Reserved for future use to encode sampling behavior, currently
+ * encoded explicitly in TraceId.sampled (Option[Boolean]).
+ */
+  val SamplingKnown = 1L << 1
+  val Sampled = 1L << 2
+```
+
+## Relationship to JMS (Java Message Service)
+
+The single header format, notably the name of "b3" solves propagation concerns a prefixed
format causes with JMS.
+
+JMS requires that message header names follow java naming conventions. Notably this excludes
hyphens and dots. For example, [in Camel](http://people.apache.org/~dkulp/camel/jms.html),
there's a naming policy to map to and from these constraints.
+
+For example, many simply downcase B3 headers to `x-b3-traceid` for use in non-http transports
(which might not be case insensitive). In JMS, this wouldn't work, even if rabbitmq is used
underneath which has no such constraint. Ex, if using camel JMS, this header would map into
`x_HYPHEN_b3_HYPHEN_traceid`.
+
+In some cases, JMS headers are replaced based on constants or other patterns, like globally
replacing hyphens with underscores. Interestingly opentracing decided on a [different pattern](https://github.com/opentracing-contrib/java-jms/pull/1)
`_$dash$_`, though they have no pattern for dots: For example, if you were using the OT library
and camel on the other, you'd get `x_HYPHEN_b3_HYPHEN_traceid` on one side and `x_$dash$_b3_$dash$_traceid`
on the other, mutually unreadable.
+
+The universal implication of using JMS is that it implies global coordination of these naming
patterns, or there will be propagation incompatibility (ex traces will restart). This will
also break anything else propagated that isn't a trace ID. This type of problem has [already
been noticed](https://github.com/spring-cloud/spring-cloud-sleuth/issues/537) in early users
of message tracing.
+
+Importantly, even if we were to have static replacements for `X-B3-` headers, we don't know
if users will introduce propagated headers with dots or hyphens in them. In other words, a
constant-based solution helps, but won't fix it.
+
+To compound issues, any mapping or heuristic approach to prefixed headers becomes virulent
and not limited to only messaging systems. Some integration services pass headers from http
directly into messaging headers. Even if it feels tempting to make a "messaging" header name,
we have to keep in mind that many are already using header names with hyphens. If we start
propagating underscores universally, we introduce complexity to existing non-JMS consumers.
+
+For all of these reasons, a policy of only using "b3" format when using JMS is the safest
policy. That implies using prefixed headers `X-B3-` is an antipattern.
diff --git a/README.md b/README.md
index 5aa6ae3..ab0be52 100644
--- a/README.md
+++ b/README.md
@@ -119,7 +119,7 @@ Note: Before this specification was written, some tracers propagated `X-B3-Sampl
 Debug is encoded as `X-B3-Flags: 1`. Debug implies an accept decision, so don't also send
the `X-B3-Sampled` header.
 
 ## Single Header
-A single header named `b3` standardized in late 2018 for use in JMS and w3c `tracestate`.
Design and rationale are captured [here](https://cwiki.apache.org/confluence/display/ZIPKIN/b3+single+header+format).
Check or update our [support page](https://cwiki.apache.org/confluence/display/ZIPKIN/b3+single+header+support)
for adoption status.
+A single header named `b3` standardized in late 2018 for use in JMS and w3c `tracestate`.
Design and rationale are captured [here](RATIONALE.md). Check or update our [status page](STATUS.md)
for adoption status.
 
 In simplest terms `b3` maps propagation fields into a hyphen delimited string.
 
diff --git a/STATUS.md b/STATUS.md
new file mode 100644
index 0000000..af8dc49
--- /dev/null
+++ b/STATUS.md
@@ -0,0 +1,44 @@
+# Status of instrumentation libraries and B3 support
+
+This is a list of zipkin libraries and their status in supporting b3 features
+
+| language   | library | 128-bit trace ID | b3 single format | notes |
+| ---------- | ------- | ---------------- | ---------------- | ----- |
+| javascript | [zipkin-js](https://github.com/openzipkin/zipkin-js) | v0.5+ [supported](https://github.com/openzipkin/zipkin-js/blob/a8ab73d26157a3b25b207425e2808ee39105afa1/packages/zipkin/README.md#usage)
| [unsupported](https://github.com/openzipkin/zipkin-js/issues/259) | |
+| python     | [py_zipkin](https://github.com/Yelp/py_zipkin) | 0.8+ [supported](https://github.com/Yelp/py_zipkin/blob/665428305ba3cadec5e6488c801bdaca12b3e311/py_zipkin/zipkin.py#L164)
| [unsupported](https://github.com/Yelp/py_zipkin/issues/98) | |
+| java       | [brave](https://github.com/openzipkin/brave) | 3.15+/4.9+ [supported/epoch128](https://github.com/openzipkin/brave/tree/master/brave-core#128-bit-trace-ids)
| 5.3+ [supported](https://github.com/apache/incubator-zipkin-brave/blob/5a17fc018613958db00cc7b6951826e95f1b9d6c/brave/src/main/java/brave/propagation/B3SingleFormat.java)
| |
+| scala      | [finagle](https://github.com/twitter/finagle) | 0.40+ downgrade | [in progress](https://github.com/twitter/finagle/pull/749)
| |
+| ruby       | [zipkin-ruby](https://github.com/openzipkin/zipkin-ruby) | 0.28+ [supported](https://github.com/openzipkin/zipkin-ruby/blob/ec875f3640dc2ce8070969f3b0c889d7b7121063/README.md#configuration-options)
| [unsupported](https://github.com/openzipkin/zipkin-ruby/issues/126) | |
+| go         | [zipkin-go](https://github.com/openzipkin/zipkin-go) | 0.1+ [supported](https://godoc.org/github.com/openzipkin/zipkin-go#WithTraceID128Bit)
| 0.1+ [supported](https://godoc.org/github.com/openzipkin/zipkin-go/propagation/b3#InjectOption)
| |
+| go         | [zipkin-go-opentracing](https://github.com/openzipkin/zipkin-go-opentracing)
| 0.2+ [supported](https://godoc.org/github.com/openzipkin/zipkin-go-opentracing#TraceID128Bit)
| unsupported | |
+| php        | [zipkin-php](https://github.com/openzipkin/zipkin-php) | 1+ [supported](https://github.com/openzipkin/zipkin-php/blob/b143bf577b5b328df656dede4f6b8aeec38b201a/src/Zipkin/TracingBuilder.php#L122)
| [unsupported](https://github.com/openzipkin/zipkin-php/issues/92) | |
+| java       | [wingtips](https://github.com/Nike-Inc/wingtips) | 0.11.2+ transparent | [unsupported](https://github.com/Nike-Inc/wingtips/issues/80)
| |
+| java       | [jaeger](https://github.com/uber/jaeger-client-java) | 0.10+ downgrade | unsupported
| | 
+| csharp     | [zipkin4net](https://github.com/openzipkin/zipkin4net) | 0.4+ supported |
[supported](https://github.com/openzipkin/zipkin4net/blob/c30d8244cad5c219f9b891a513b0032f0047a2fd/Src/zipkin4net/Src/Propagation/B3SingleFormat.cs)
| | 
+
+## 128-bit trace ID
+
+A 128-bit trace ID is when `X-B3-TraceId` has 32 hex characters as opposed to 16. For example,
`X-B3-TraceId: 463ac35c9f6413ad48485a3953bb6124`.
+
+Here are the status options for 128bit `X-B3-TraceId`:
+* unsupported: 32 character trace ids will break the library
+* downgrade: Read 32 character trace ids by throwing away the high bits (any characters left
of 16 characters). This effectively downgrades the ID to 64 bits.
+* transparent: Pass `X-B3-TraceId` through the system without interpreting it.
+* supported:  Can start traces with 128-bit trace IDs, propagates and reports them as 128-bits
+* epoch128: Can start traces with 128-bit trace IDs which are prefixed with epoch seconds
+
+### epoch128
+When a trace ID is 128-bits and the first 32 bits are epoch seconds, the ID can be translated
into
+an Amazon Web Services ID. Tracers who do this can tunnel through ELBs, for example.
+
+Here's an example implementation
+```java
+  static long nextTraceIdHigh(Random prng) {
+    long epochSeconds = System.currentTimeMillis() / 1000;
+    int random = prng.nextInt();
+    return (epochSeconds & 0xffffffffL) << 32
+        |  (random & 0xffffffffL);
+  }
+```
+
+See openzipkin/zipkin#1754


Mime
View raw message