incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: Live Metrics & Metrics(raw) Pages
Date Fri, 17 May 2013 02:26:11 GMT
Ok, so hear are my thoughts on the status pages.  Anyone else can jump in
here as well.

Things that are old and possibly/probably need to be removed.

org.apache.blur.gui.HeapMetrics
org.apache.blur.gui.QueryMetrics
org.apache.blur.gui.SystemLoadMetrics

And any corresponding servlets that serve these old metrics.

The "new" metrics can be accessed from the
com.yammer.metrics.reporting.MetricsServlet which is mapped to /metrics in
the HttpJettyServer.  All of these classes and mappings can be found in the
blur-gui project.

At this point I think the best route is to get all of the metrics to be
displayed in a nice easy to read format on a single page.  The current
architecture of the http status server is just serving up simple servlets
and jsps which is not the most modern technologies but they don't require
any additional libraries.  I'm not an expert in web frameworks, but I think
a very simple approach would be to use jquery to ajax in the metrics from
/metrics and then build up a nice table for them in javascript or something
like that.  Also I'm not opposed to bringing in other libraries or going
with a different approach, but since they are just status pages I would
like to keep them as simple as possible.

In short I think that if you use you best judgement on them they will
be infinitely better than the currently are (which is broken).  :)  Thanks!

Aaron


On Thu, May 16, 2013 at 5:08 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Sure Aaron! Just give me an overview.
> I made a few minor changes to bring the pages up (json data is not in the
> proper format...and few broken urls).
>
> - Rahul
>
>
> On Thu, May 16, 2013 at 1:52 PM, Aaron McCurry <amccurry@gmail.com> wrote:
>
> > If you have some thoughts or would like to help fix them I can try and
> walk
> > you through what's in the Metrics library as far as reporters and how we
> > could hook things up.  Thanks!
> >
> > Aaron
> >
> >
> > On Thu, May 16, 2013 at 4:50 PM, Aaron McCurry <amccurry@gmail.com>
> wrote:
> >
> > >
> > > On Thu, May 16, 2013 at 2:24 AM, rahul challapalli <
> > > challapallirahul@gmail.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> On the live metrics page(up and running), Blur Queries / Fetches /
> > Mutate
> > >> Requests and Blur Fetches  / Mutate Record Rates reports are not
> showing
> > >> any data. I created a table, loaded data, and also queried the table.
> I
> > >> checked the returned json data and it is in the expected format but
> the
> > >> values are all 0. JVM Heap and System load reports are displaying
> proper
> > >> data.
> > >>
> > >
> > > My guess is that the status pages are still broken from the codehaus
> > > Metrics library upgrade.
> > >
> > >
> > >>
> > >> Metrics(raw)  : The page is up but data is not being read
> properly.Below
> > >> is
> > >> a sample json data returned by the MetricsServlet. How do we intend to
> > >> capture this data on page ("jvm", and "org.apache.blur.Blur.Thrift
> Calls
> > >> in µs"
> > >> are in the top level)
> > >>
> > >
> > > I would say that the metrics pages across the board need to be
> reworked.
> > >
> > >
> > >>
> > >> {
> > >>    "jvm":{
> > >>       "vm":{
> > >>          "name":"Java HotSpot(TM) 64-Bit Server VM",
> > >>          "version":"1.6.0_45-b06-451-11M4406"
> > >>       },
> > >>       "memory":{
> > >>          "totalInit":2.4317952E7,
> > >>          "totalUsed":4.5249872E7,
> > >>          "totalMax":1.19668736E9,
> > >>          "totalCommitted":1.12070656E8,
> > >>          "heapInit":0.0,
> > >>          "heapUsed":1.9243392E7,
> > >>          "heapMax":1.06037248E9,
> > >>          "heapCommitted":8.5000192E7,
> > >>          "heap_usage":0.01814776633961681,
> > >>          "non_heap_usage":0.19078656710111178,
> > >>          "memory_pool_usages":{
> > >>             "CMS Old Gen":0.0035037057740347727,
> > >>             "CMS Perm Gen":0.2833598067120808,
> > >>             "Code Cache":0.032642364501953125,
> > >>             "Par Eden Space":0.14699513505144818,
> > >>             "Par Survivor Space":0.011420156441482842
> > >>          }
> > >>       },
> > >>       "daemon_thread_count":38,
> > >>       "thread_count":43,
> > >>       "current_time":1368683167699,
> > >>       "uptime":348,
> > >>       "fd_usage":0.01640625,
> > >>       "thread-states":{
> > >>          "timed_waiting":0.4418604651162791,
> > >>          "new":0.0,
> > >>          "terminated":0.0,
> > >>          "runnable":0.16279069767441862,
> > >>          "waiting":0.3953488372093023,
> > >>          "blocked":0.0
> > >>       },
> > >>       "garbage-collectors":{
> > >>          "ParNew":{
> > >>             "runs":13,
> > >>             "time":100
> > >>          },
> > >>          "ConcurrentMarkSweep":{
> > >>             "runs":0,
> > >>             "time":0
> > >>          }
> > >>       }
> > >>    },
> > >>    "org.apache.blur.Blur.Thrift Calls in µs":{
> > >>       "cancelQuery":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "configuration":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "controllerServerList":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "createTable":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "currentQueries":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "describe":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "disableTable":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "enableTable":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "fetchRow":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "getTableStats":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "isInSafeMode":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "mutate":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "mutateBatch":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "optimize":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "query":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "queryStatusById":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "queryStatusIdList":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "recordFrequency":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "removeTable":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "schema":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "shardClusterList":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "shardServerLayout":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "shardServerLayoutState":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "shardServerList":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "tableList":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "tableListByCluster":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "tableStats":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       },
> > >>       "terms":{
> > >>          "type":"histogram",
> > >>          "count":0,
> > >>          "min":0.0,
> > >>          "max":0.0,
> > >>          "mean":0.0,
> > >>          "std_dev":0.0,
> > >>          "median":0.0,
> > >>          "p75":0.0,
> > >>          "p95":0.0,
> > >>          "p98":0.0,
> > >>          "p99":0.0,
> > >>          "p999":0.0
> > >>       }
> > >>    }
> > >> }
> > >>
> > >> - Rahul
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message