htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ayola Jayamaha <raphaelan...@gmail.com>
Subject Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
Date Fri, 24 Apr 2015 09:24:37 GMT
Hi All,

Thanks for your responses.

On Fri, Apr 24, 2015 at 9:43 AM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> I wonder if the htrace UI backend could be defined as an interface, and an
> implementation be provided that goes against the Phoenix table. Further, if
> the UI is modular, Phoenix can embed it into it's own management service,
> should such a service surface.
>
> Actually, I think my experience with Calcite/Avatica and the Phoenix Query
> Server is a nice example. The upstream library project (HTrace) can provide
> a modular implementation with well defined interfaces and an out-of-the-box
> implementation. Those modules can be used by downstream projects (Phoenix)
> to drive their own specific needs if the out-of-the-box implementations are
> insufficient or can otherwise be adapted. Users of HDFS but not Phoenix
> should enjoy a nice UI for visualizing traces, just like HBase and Phoenix.
> Having that core code living in HTrace makes good sense.
>

My solution includes a dashboard for visualizing the trace information
which would be accessed via querying the Phoenix Table SYSTEM.TRACING_STATS.


> However, I'm concerned that our aspirations of cross-project reuse may be
> a hindrance for Ayola's GSoC requirements. I think she'll have a more
> fruitful experience if there's a concrete, single-project deliverable. With
> that in mind, I'm inclined to suggest she "borrow" HTrace's efforts as
> she's able, to facilitate Phoenix working out it's precise needs. This
> keeps her deliverables focused and of direct value to the sponsoring
> project. Unless, that is, HTrace is ready with a modular design and it's UI
> elements can be packaged as library components today.
>
> -n
>
> On Wed, Apr 22, 2015 at 1:45 PM, Stack <stack@duboce.net> wrote:
>
>> On Wed, Apr 22, 2015 at 1:16 PM, James Taylor <jamestaylor@apache.org>
>> wrote:
>>
>>> On Wed, Apr 22, 2015 at 12:52 PM, Stack <stack@duboce.net> wrote:
>>> > On Wed, Apr 22, 2015 at 12:41 PM, James Taylor <jamestaylor@apache.org
>>> >
>>> > wrote:
>>> >>
>>> >> bq. So a web UI in HTrace itself would help more people than a web UI
>>> >> in just Phoenix
>>> >> You trying to poach our GSoC student, Colin? :-)
>>> >>
>>> >> The end goal is to provide a UI to visualize performance
>>> >> characteristics of an HBase cluster running Phoenix queries.
>>> >
>>> >
>>> > Where are you thinking this UI should live James? (Where are you
>>> thinking
>>> > the traces should reside? Would you want to query the traces table
>>> using
>>> > Phoenix?)
>>>
>>> UI would live in Phoenix and be part of the distro. You'd likely want
>>> to be able to issue queries (maybe embed sqlline or some other SQL
>>> client). I think it'd be Phoenix/SQL-specific, but it's possible that
>>> shared UI components could live in HTrace.
>>>
>>> Yes, we have a custom sink that persists the trace information into a
>>> Phoenix table to allow adhoc querying. The idea would be that the UI
>>> would query that table. See http://phoenix.apache.org/tracing.html.
>>> You can play around with this today - it's available in our 4.3.1
>>> release (including turning tracing on and off through sqlline).
>>>
>>>
>> So, for this to work on any random cluster, HDFS, HBase, and Phoenix will
>> all be configured to write traces to a Phoenix table.
>>
>> The UI will do (or does) SQL against this table or does it use HBase APIs?
>>
>> You would not be interested in deferring 'trace' to an external system?
>> You are more interested in leveraging bits and pieces of htrace to host
>> your own?
>>
>> For visualizing tracing information I would be mainly using
the SYSTEM.TRACING_STATS table. Which will be updated when Hadoop matrics2
sink writes the matrics to the said table using HTrace library and HBase
tracing utilities.

If the required matrices are not enough to cater the needs such as time
sliding facility, tracing tree view with parent/child relations... etc, I
would write my own sinks/writers/classes to cater for my requirements.


>


-- 
Best Regards,
Ayola Jayamaha
http://ayolajayamaha.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message