hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-20234) Expose in-memory compaction metrics
Date Thu, 05 Apr 2018 15:42:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-20234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427136#comment-16427136

stack commented on HBASE-20234:

bq. There are currently no per-store counters that I have described (number of in-memory-flushes,
number of saved bytes), nor that they make it up to region-level. No problem to add them.

[~anastas] Thanks.

Be careful on per-store counters. HBase is already crippled keeping counts (one of the main
users of CPU). The per-region counters are already costly. If we were to do per-store, the
number of counts would blossom.

bq. Alternatively we can coordinate the collection of the in-memory-flush counters to the
flush-to-disk update, that already exists in the code. 

If no flush to disk, no counters. If no flush to disk, then we are not doing heavy writes.
Maybe its ok to do this. It'd be better than a background thread doing counter updates. When
hbase is not doing counters, its context switching between all the various background threads
that do caretaking; no cpu left over to do actual work (smile).

Thanks for looking at this A. Keep asking questions. The metrics system is cryptic. Lets save
you getting lost in it if we can help it.

> Expose in-memory compaction metrics
> -----------------------------------
>                 Key: HBASE-20234
>                 URL: https://issues.apache.org/jira/browse/HBASE-20234
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: stack
>            Assignee: Anastasia Braginsky
>            Priority: Major
> Hard to glean insight from how well in-memory compaction is doing currently. It dumps
stats into the logs but better if they were available to a dashboard. This issue is about
exposing a couple of helpful counts. There are already by-region metrics. We can add a few
for in-memory compaction (Help me out [~anastas]... what counts would be best to expose).
> Flush related metrics include....
> {code}
> Namespace_default_table_tsdb-tree_region_cfbf23e7330a1a2bbde031f9583d3415_metric_flushesQueuedCount:
> description: "Number flushes requested/queued for this region",
> value: 0
> {
> description: "The number of cells flushed to disk",
> value: 0
> },
> {
> description: "The total amount of data flushed to disk, in bytes",
> value: 0
> },
> ...
> {code}

This message was sent by Atlassian JIRA

View raw message