hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15566) Remove HTrace support
Date Mon, 30 Jul 2018 22:39:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562619#comment-16562619

stack commented on HADOOP-15566:

bq. I just think we're going to be hard-pressed to make an informed decision without pairings
of trace visualizations (ideally in many tracing systems to illustrate portability) and the
respective instrumentation code to illustrate non-bloat / maintainability.... stack you were
suggesting we try this on dev – any pointers to a non-HDFS / non-HBase expert for a place
to focus on for such an exercise?

Yeah. I just started a DISCUSS thread that points here up on dev-common. Hopefully, we'll
attract doers/volunteers.

What you thinking [~bensigelman]? You (or your company) running a compare of libs -- OT/OC/Hacked
HTrace -- for a neutral party/volunteer to evaluate?

bq. ....I wonder if it's would be worth evaluating writing a htrace-api->opentracing-java
or htace-api->census or htrace-api->zipkin...

I just did a refresher and unfortunately it'd be a bit of awkward work to do [~michaelsembwever].
HTrace core entities -- probably the font of friction (We'd have to check; we could for sure
do some fixup around when no trace enabled) -- are classes rather than Interfaces and do work
passing Spans though no trace enabled. The other awkward fact is that there are two htrace
APIs afloat in Hadoop currently, an htrace3 in older Hadoops and an htrace4 (though in different

Getting traces into zipkin though should be easy enough. htrace dumps to spanreceiver implementations
and these are easy to write and plugin.

[~bogdandrutu] Thanks boss for the OC input. The local-view (z-pages) makes sense. Nice instrumentation
example over in the hbase client for talking to (cloud) bigtable too (smile) -- https://github.com/GoogleCloudPlatform/cloud-bigtable-client.

> Remove HTrace support
> ---------------------
>                 Key: HADOOP-15566
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15566
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 3.1.0
>            Reporter: Todd Lipcon
>            Priority: Major
>              Labels: security
>         Attachments: Screen Shot 2018-06-29 at 11.59.16 AM.png, ss-trace-s3a.png
> The HTrace incubator project has voted to retire itself and won't be making further releases.
The Hadoop project currently has various hooks with HTrace. It seems in some cases (eg HDFS-13702)
these hooks have had measurable performance overhead. Given these two factors, I think we
should consider removing the HTrace integration. If there is someone willing to do the work,
replacing it with OpenTracing might be a better choice since there is an active community.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message