hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10158) Add sync rate histogram to HLogPE
Date Fri, 11 Apr 2014 22:57:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967212#comment-13967212
] 

Nick Dimiduk commented on HBASE-10158:
--------------------------------------

Did you intend to post a patch?

> Add sync rate histogram to HLogPE
> ---------------------------------
>
>                 Key: HBASE-10158
>                 URL: https://issues.apache.org/jira/browse/HBASE-10158
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance, wal
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.99.0
>
>
> HLogPE emits a few metrics that help comparing WAL implementations -- sync and append
counts.  This patch adds a histogram around time between sync calls to ensure implementations
are not clumping sync invocations.  Helps comparing implementations.
> It also makes it so the just committed HBASE-8755 code calls postSync and postAppend
methods.
> [~eclark] Mind reviewing?  Also, are these additions actually needed?  Could I dump the
metrics we already collect if I only did proper startup invocation; i.e. use our metrics rather
than these yammer ones?
> Here is what the output looks like on a run of HLogPE:
> {code}
> ...
> 2013-12-13 09:41:15,641 INFO  [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] hfile.CacheConfig:
Allocating LruBlockCache with maximum size 395 M
> 2013-12-13 09:41:15,652 INFO  [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] compactions.CompactionConfiguration:
size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000;
throttle point 2684354560; delete expired; major period 604800000, major jitter 0.500000
> 2013-12-13 09:41:15,662 INFO  [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] util.ChecksumType:
Checksum using org.apache.hadoop.util.PureJavaCrc32
> 2013-12-13 09:41:15,663 INFO  [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] util.ChecksumType:
Checksum can use org.apache.hadoop.util.PureJavaCrc32C
> 2013-12-13 09:41:15,666 DEBUG [main] regionserver.HRegion: Found 0 recovered edits file(s)
under file:/Users/stack/checkouts/trunk/target/test-data/a62eb348-138b-4275-b33a-12ebb12c20b9/HLogPerformanceEvaluation/data/default/HLogPerformanceEvaluation/a77539a833048fa441d3195c853cf6ea
> 2013-12-13 09:41:15,669 INFO  [main] regionserver.HRegion: Onlined a77539a833048fa441d3195c853cf6ea;
next sequenceid=1
> 12/13/13 9:42:15 AM ============================================================
> org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation:
>   appendMeter:
>              count = 1173705160
>          mean rate = 19180.80 bytes/ms
>      1-minute rate = 16437.82 bytes/ms
>      5-minute rate = 12716.38 bytes/ms
>     15-minute rate = 11783.97 bytes/ms
>   syncMeter:
>              count = 1726060
>          mean rate = 28.20 syncs/ms
>      1-minute rate = 24.16 syncs/ms
>      5-minute rate = 18.68 syncs/ms
>     15-minute rate = 17.30 syncs/ms
> org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation.nanos-between-syncs:
>   syncHistogram:
>                min = 13000.00
>                max = 32438000.00
>               mean = 34754.07
>             stddev = 38993.58
>             median = 29000.00
>               75% <= 37000.00
>               95% <= 56000.00
>               98% <= 84420.00
>               99% <= 99000.00
>             99.9% <= 267709.00
> 2013-12-13 09:42:25,040 INFO  [t0] wal.HLogPerformanceEvaluation: t0 took 69.369s 28831.320ops/s
> 2013-12-13 09:42:25,040 INFO  [main] wal.HLogPerformanceEvaluation: Summary: threads=1,
iterations=2000000, syncInterval=0 took 69.369s 28831.320ops/s
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message