htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@apache.org>
Subject Re: [DISCUSS] Attic podling Apache HTrace?
Date Fri, 22 Sep 2017 04:39:11 GMT
On Wed, Sep 13, 2017, at 09:01, Mike Drob wrote:
> Hi folks,
> 
> Want to pull this conversation back into people's minds to summarize and
> reposit some questions.
> 
> It looks like there is consensus that 1) HTrace is generally useful but
> 2)
> needs to refocus on more specific use cases rather than trying to be a
> general purpose tool and 3) suffers from being niche on niche.
> 
> There have been lots of folks volunteering to contribute, but I don't see
> any resulting JIRA or mailing list activity. We all can have the best of
> intentions while still not having the necessary time available to follow
> through, myself included.
> 
> Projects can still be successful outside of the ASF; retiring as a
> podling
> does not preclude others from picking up the works. As one example, I
> have
> seen many vibrant communities run as github projects with a core set of
> dedicated contributors. I don't believe that living in the ASF makes it
> any
> easier or harder for other Apache projects such as Hadoop to integrate
> with
> us, so long as the licensing remains compatible. In the future, if a
> diverse community springs up again and finds the desire to come back, I
> don't think that will be a problem to re-incubate at that point either.

Hi Mike,

I'm on vacation this week, so I apologize for the brevity (and delayed
responses).

I think it would make more sense to fold HTrace into Hadoop if we wanted
to disband the podling.  At a minimum, there is value in allowing Hadoop
to use the trace systems that are out there (Zipkin, OpenTracing, etc.)
in a software-configurable way.

But I still hope we can find people interested in moving tracing forward
in hadoop. Let's circle back on this in a little bit...

Colin


> 
> Thoughts?
> Mike
> 
> 
> On Thu, Aug 31, 2017 at 3:32 AM, Raam Rosh-Hai <raam@findhotel.net>
> wrote:
> 
> > Thank you Colin for your response,
> > I was actually referring to Adrian's comment on making Htrace and Zipkin
> > play more nicely together, I have been using Htrace with zipkin for a while
> > now and I am doing pretty well but I was wondering if there are some
> > glaring pain points that should be attended.
> >
> > On 30 August 2017 at 23:41, Colin McCabe <cmccabe@apache.org> wrote:
> >
> > > On Mon, Aug 21, 2017, at 02:09, Raam Rosh-Hai wrote:
> > > > I tend to agree that zipkin should be used as a frontend and I am
> > pretty
> > > > sure I will have some time to advance that in the coming weeks if one
> > of
> > > > the experts could create a few tasks.
> > >
> > > Hi Raam,
> > >
> > > If you're interested in this, check out the Zipkin trace sink.  That
> > > allows you to send HTrace spans to Zipkin.
> > >
> > > >
> > > > I think no one touched the obvious thing that tracing lacks any kind of
> > > > hype, it's just not glamours and in order to want to use such a
> > framework
> > > > you need to be versed in this world and to actually suffer from
> > > > distributed
> > > > debugging.
> > >
> > > Yeah, I think this is an area where some publicity and outreach would be
> > > a good thing :)
> > >
> > > It would help a lot if we could identify some core use-cases, like Stack
> > > said, and make sure everything works great for those.
> > >
> > > Colin
> > >
> > >
> > > >
> > > > so yes, I think this project deserves one last push before putting it
> > in
> > > > the attic and I would be happy to complete a few issues in order to
> > make
> > > > this happen.
> > > >
> > > >
> > > >
> > > > On 19 August 2017 at 03:55, Adrian Cole <adrian.f.cole@gmail.com>
> > wrote:
> > > >
> > > > > > Thanks Adrian for the editorial on the landscape. Helps, especially
> > > > > coming
> > > > > > from yourself.
> > > > > we aim to please
> > > > >
> > > > > > Given current state of the project, a retrofit to come up on
OT is
> > > not
> > > > > the
> > > > > > solution to the topic-at-hand (and besides I have a colored
opinion
> > > on
> > > > > > taking on the API of another after spending a bunch of time
> > recently
> > > > > > undoing our mistake letting third-party Interfaces and Classes
show
> > > > > through
> > > > > > in hbase).
> > > > > sensible for any api highly disconnected from the ecosystem,
> > > > > especially without practice yet.
> > > > >
> > > > > > I appreciate the higher-level point made by Andrew, that it
is hard
> > > to
> > > > > > thread a cross-cutting library across the Hadoop landscape whether
> > > > > because
> > > > > > releases happen on the geologic time scale or that there is
little
> > > by way
> > > > > > of coordination.
> > > > > I think this is indeed leading a path towards focus, eg the H in
> > > Htrace :)
> > > > >
> > > > > > Can we do a focused 'win' like Colin suggests? E.g. hook up
hbase
> > and
> > > > > hdfs
> > > > > > end-to-end with connection to a viewer (zipkin? Or text dumps
in a
> > > > > > webpage?). A while back I had a go at the hbase side but it
was
> > > burning
> > > > > up
> > > > > > the hours just getting it hooked up w/ tests to scream if any
spans
> > > were
> > > > > > broken in a refactor. I had to put it aside.
> > > > > Incidentally, I wouldn't necessarily say Zipkin is ready out of box
> > > > > because htrace UI and query is more advanced (in some ways due to
> > some
> > > > > data storage options we have available). So, something like this
> > could
> > > > > be a move of focus which would require investment on the other side
> > to
> > > > > avail features needed, or discuss how to upgrade into them (ex if
> > > > > using hbase storage, certain queries would work). It is fair to say
> > > > > zipkin has  a great devops pipeline, we are good at fixing things.
At
> > > > > the same time, we are imperfect in impl and inexperienced in hadoop
> > > > > ecosystem. Having some way to join together could be really
> > > > > beneficial, at the cost of up-front effort (due to model, UI and
> > > > > storage differences). I would be happy to direct time, though would
> > > > > need some help because of my irrelevance in the data services space
> > > > > (something this might correct!)
> > > > >
> > > > > > Like the rest of you, my time is a little occupied elsewhere
these
> > > times
> > > > > so
> > > > > > I can't revive the project, not at the moment at least.
> > > > > ack
> > > > >
> > >
> >

Mime
View raw message