hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HBASE-4089 & HBASE-4147 - on the topic of ops output
Date Fri, 29 Jul 2011 17:32:18 GMT
On Fri, Jul 29, 2011 at 10:15 AM, Andrew Purtell <apurtell@apache.org>wrote:

> There is also the matter of HServerLoad and how that is used by the shell
> and master UI to report on cluster status.
>
> I'd like the shell to be able to let the user explore all of these
> different reports interactively.
>

+1. In some use cases the users of HBase don't have access to the log files.

With my above idea of adding an interface, we could have an implementation
which keeps the most recent report in memory and exposes it via a servlet as
well.


>
> At the very least, they should all be handled the same way.
>
> And then there is Riley's work over at FB on a slow query log. How does
> that fit in?
>
> Best regards,
>
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
> >________________________________
> >From: Todd Lipcon <todd@cloudera.com>
> >To: dev@hbase.apache.org
> >Sent: Friday, July 29, 2011 9:58 AM
> >Subject: Re: HBASE-4089 & HBASE-4147 - on the topic of ops output
> >
> >What I'd prefer is something like:
> >
> >interface BlockCacheReportSink {
> >  public void reportStats(BlockCacheReport report);
> >}
> >
> >class LoggingBlockCacheReportSink {
> >  ... {
> >    log it with whatever formatting you want
> >  }
> >}
> >
> >then a configuration which could default to the logging implementation,
> but
> >orgs could easily substitute their own implementation. For example, I
> could
> >see wanting to do an implementation where it keeps local RRD graphs of
> some
> >stats, or pushes them to a central management server.
> >
> >The assumption is that BlockCacheReport is a fairly straightforward
> "struct"
> >with the non-formatted information available.
> >
> >-Todd
> >
> >On Fri, Jul 29, 2011 at 4:15 AM, Doug Meil <doug.meil@explorysmedical.com
> >wrote:
> >
> >>
> >> Hi Folks-
> >>
> >> You probably already my email yesterday on this...
> >>  https://issues.apache.org/jira/browse/HBASE-4089 (block cache report)
> >>
> >> ...and I just created this one...
> >>  https://issues.apache.org/jira/browse/HBASE-4147 (StoreFile query
> >> report)
> >>
> >> What I'd like to run past the dev-list is this:  if Hbase had periodic
> >> summary usage statistics, where should they go?  What I'd like to throw
> >> out for discussion is that I'm suggesting that it should simply go to
> the
> >> log files and users can slice and dice this on their own.  No UI (I.e.,
> >> JSPs), no JMX, etc.
> >>
> >>
> >> The summary out the output is this:
> >> BlockCacheReport:  on configured interval, print out summary of
> blockcache
> >> (at table/CF level) to log file. This one is point-in-time, not delta.
> >>
> >> StoreFile read report:  on configured interval, print out summary of
> >> StoreFile accesses and how much time was spent reading each StoreFile to
> >> log file.
> >>
> >> Thoughts?
> >>
> >> Doug
> >>
> >> >
> >>
> >>
> >
> >
> >--
> >Todd Lipcon
> >Software Engineer, Cloudera
> >
> >
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message