accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: [DISCUSS] tracing framework updates
Date Tue, 27 Feb 2018 20:15:13 GMT
Wow... that's, erm, quite the paper. Nothing like taking some pot-shots 
at another software project and quoting folks out of context.

Does it help to break down the problem some more?

* Is Accumulo getting benefit from tracing its library?
* Is Accumulo getting benefit from tracing context including HDFS calls?

I feel like it is a nice tool to have in your toolbelt (having used it 
successfully in the past), but I wonder if it's the most effective thing 
to keep inside of Accumulo. Specifically, would it be better to just 
pull this out of Accumulo outright?

I don't think I have an opinion yet.

On 2/27/18 1:08 PM, Ed Coleman wrote:
> For general discussion - Facebook recently (Oct 28, 2017) published a paper on tracing:
Canopy: An End-to-End Performance Tracing and Analysis System (https://research.fb.com/publications/canopy-end-to-end-performance-tracing-at-scale/)
> 
> As a bonus, they referenced Accumulo and HTrace in section 2.2
> 
> "Mismatched models affected compatibility between mixed system versions; e.g. Accumulo
and Hadoop were impacted by the “continued lack of concern in the HTrace project around
tracing during upgrades”
> 
> 
> -----Original Message-----
> From: Tony Kurc [mailto:tkurc@apache.org]
> Sent: Tuesday, February 27, 2018 12:57 PM
> To: dev@accumulo.apache.org
> Subject: Re: [DISCUSS] tracing framework updates
> 
> I have some experience with opentracing, and it definitely seems promising, however,
potentially promising in the same way htrace was... That being said, I did a cursory thought
exercise of what it would take to do a swap of the current tracing in accumulo to opentracing,
and I didn't come across any hard problems, meaning it could be a fairly straightforward refactor.
I was hoping to explore the community a bit more at some upcoming conferences
> 
> On Feb 27, 2018 11:59 AM, "Sean Busbey" <busbey@apache.org> wrote:
> 
>>
>>
>> On 2018/02/27 16:39:02, Christopher <ctubbsii@apache.org> wrote:
>>> I didn't realize HTrace was struggling in incubation. Maybe some of
>>> us
>> can
>>> start participating? The project did start within Accumulo, after all.
>> What
>>> does it need? I also wouldn't want to go back to maintaining cloudtrace.
>>>
>>
>> I suspect it's too late for HTrace. The last commit to the main
>> development branch was May 2017. They had a decent run of activity in
>> 2015 and an almost-resurgence in 2016, but they never really got
>> enough community traction to survive the normal ebb and flow of
>> contributor involvement.
>>
>> They need the things any project needs to be sustainable: regular
>> release cadences, a responsive contribution process, and folks to do
>> the long slog of building interest via e.g. production adoption.
>>
>>> I'm unfamiliar with OpenTracing, but it was my understanding that
>>> Zipkin was more of a tracing sink, than an instrumentation API.
>>> HTrace is
>> actually
>>> listed as an instrumentation library for Zipkin (among others).
>>>
>>
>> I think the key is that for a instrumentation library to get adoption
>> it needs a good sink that provides utility to operators looking to
>> diagnose problems. It took too long for HTrace to provide any tooling
>> that could help with even simple performance profiling. Maybe hooking
>> it into Zipkin would get around that. Personally, I never managed to
>> get the two to actually work together.
>>
>> My listing Zipkin as an option merely reflects my prioritization of
>> practical impact of whatever we go to. I don't want to adopt some
>> blue-sky effort. FWIW, OpenTracing docs at least claim to also provide
>> a zipkin-sink compatible runtime.
>>
>> There's a whole community that just does distributed monitoring, maybe
>> someone has time to survey some spaces and see if OpenTracing has any legs.
>>
> 

Mime
View raw message