htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
Subject Re: HTrace for Nutch 2.x Search Stack
Date Fri, 27 Feb 2015 21:31:50 GMT
Thanks ll for the info here.
I am going to press forward with investigating HTrace for the Nutch 2.X
Search Stack.
Lewis


On Fri, Feb 27, 2015 at 11:58 AM, Colin P. McCabe <cmccabe@apache.org>
wrote:

> Hi Lewis,
>
> Good questions.  I would say HTrace differs from TRACE logging (or
> other single-node metrics, JMX, audit logs, etc.) in that it pulls
> together information from across the cluster.  This is something that
> is a major pain point when using a distributed system such as HDFS.
> Just to diagnose a slow write, you might have to match up logs from a
> client log and the logs of 3 different datanodes.  The big idea behind
> htrace is two things: integrating those logging sources, and using
> sampling to instrument performance in production.  The main thing
> htrace deals with is "spans" which are lengths of time.
>
> We're working on a web UI that will allow people to search for spans
> by time, duration, and name (among other things).  It's not quite
> finished now (hoping to have something usable in HTrace 3.2.0 or maybe
> 3.3.0... but abe can comment more on that.)
>
> Here's an early screenshot (probably way out of date now):
>
> https://issues.apache.org/jira/secure/attachment/12689757/Search%20page%20skeleton%20-%200.png
>
> There is also a plan to create a visualization of parent/child
> relationships on the web UI, by using the d3 library (which can draw
> graphs, and do many other things besides.)
>
> In the meantime, there's an option to product a graphviz file from a
> file containing span JSON.  That way you can draw a graph of
> parent/child relationships with the "dot" tool, available on Linux.
> Uh... unfortunately it's broken right now... let me file a JIRA for
> that :P  This is a very new feature, got added earlier this week.
>
> The web UI is a great place to get involved right now... there is a
> lot of work going on there and we've been adding new contributors.
>
> Colin
>
> On Thu, Feb 26, 2015 at 1:46 PM, Lewis John Mcgibbney
> <lewis.mcgibbney@gmail.com> wrote:
> > Hi Nick,
> >
> > Grand. Thank you
> >
> > What is visualization looking like right now? It there currently a
> > mechanism for visualizing HTrace structures?
> > Is it worth considering posting something like this as a GSoC project is
> > one does not currently exist?
> > Thanks
> > Lewis
> >
> >
> > On Thu, Feb 26, 2015 at 1:31 PM, Nick Dimiduk <ndimiduk@gmail.com>
> wrote:
> >
> >> Hi Lewis,
> >>
> >>
>



-- 
*Lewis*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message