hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15464) Flush / Compaction metrics revisited
Date Fri, 18 Mar 2016 02:10:33 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Enis Soztutar updated HBASE-15464:
    Attachment: hbase-15464_v3.patch

Thanks Elliott for taking a look. 

Addressed the test failure and findbugs issue. 
The changes related to compaction throttle is just a cleanup of {{compact(ThroughputController
throughputController)}} version of the method since we have {{compact(ThroughputController
throughputController, User user)}} that replaced it some time ago. 

> Flush / Compaction metrics revisited
> ------------------------------------
>                 Key: HBASE-15464
>                 URL: https://issues.apache.org/jira/browse/HBASE-15464
>             Project: HBase
>          Issue Type: Sub-task
>          Components: metrics
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0, 1.3.0, 1.4.0
>         Attachments: hbase-15464_v1.patch, hbase-15464_v2.patch, hbase-15464_v3.patch
> We can add a couple of metrics related to flushes and compactions: 
>  - flush memstore and output file size histogram: This will allow seeing whether we are
flushing too early due to memory pressure, too many regions, etc. Tracking flush memstore
size vs output file size is useful in understanding the block encoding compression benefits.

>  - total flushed output bytes: This will allow to monitor the IO / throughput from flushers.
You can use this to set num flushers, flush throttle, etc. 
>  - smallCompactionQueueLength / large...: This is tracked, but not emitted anymore due
to a bug. 
>  - compaction time histogram: similar to flush time histogram, how long compactions are
>  - compaction input num files / output num files histogram: How many files on average
we are compacting. Stripe compaction / date tiered compaction can use the num output files
>  - compaction input / output data sizes histogram: How much data on average we are compacting.

>  - compaction input / output total bytes: Measure compaction IO / throughput. measure
write amplification, enables to set compaction throttle. 
>  - Breakdown for above for major compactions

This message was sent by Atlassian JIRA

View raw message