hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Young (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15566) Remove HTrace support
Date Tue, 31 Jul 2018 05:43:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563157#comment-16563157

Ted Young commented on HADOOP-15566:

Hi there, I work on the OpenTracing project w/ Ben, thought I would weigh in!

I feel like there is somewhat an apples to oranges comparison going on here. To clarify what
we are trying to do with OpenTracing:
 * the instrumentation API should be an abstract interface, and should not expose implementation
details. That's the whole point, it's not about additional features.
 * The fact that some clients ship with nifty features, such as z-pages, is actually an argument
FOR an abstract interface, not against it. You can easily put a client with z-pages (or whatever
new feature comes next) behind an abstract interface. Arguing that abstraction should be abandoned
because a particular implementation has a useful feature doesn't make any sense. This no
different than LightStep or any other vendor arguing that you should bake in their tracing
client because it has a special feature. It's a form of implementation lock-in, which is
easily avoided. The whole reason we've been working on an abstracted interface for the past
several years is to decouple these choices. So it's not either/or. Use a good client behind
an abstraction, that's all.
 * Likewise with a wire protocol. I also support the w3c protocol under development. But it
is most definitely still under development. The v00 prototype version is still being mutated,
and we haven't even had a meeting yet to compare notes about initial implementations. What
would be the point in adding any instrumentation code which baked in something in this state?
It's better to use this - or any other wire protocol that the users of a hadoop may want
to use - behind an interface which allows them to swap it out without rewriting code. This
includes swapping in future versions of the w3c headers.


Again, just to reiterate: arguments about how particular clients may expose data usefully
- or otherwise have special additional features - and arguments about the benefits of one
wire protocol vs another, are actually arguments FOR an abstract instrumentation API. You
really want these choices decoupled. Better implementation details may exist tomorrow, and
the versioning/packaging of a tracing subsystem should be orthogonal to the versioning of
Hadoop itself.

Hope that adds some clarity! FWIW, I wrote a longer-form version of of my thinking here a
couple months ago, if you want more detail: [https://opensource.com/article/18/5/distributed-tracing]

> Remove HTrace support
> ---------------------
>                 Key: HADOOP-15566
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15566
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 3.1.0
>            Reporter: Todd Lipcon
>            Priority: Major
>              Labels: security
>         Attachments: Screen Shot 2018-06-29 at 11.59.16 AM.png, ss-trace-s3a.png
> The HTrace incubator project has voted to retire itself and won't be making further releases.
The Hadoop project currently has various hooks with HTrace. It seems in some cases (eg HDFS-13702)
these hooks have had measurable performance overhead. Given these two factors, I think we
should consider removing the HTrace integration. If there is someone willing to do the work,
replacing it with OpenTracing might be a better choice since there is an active community.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message