hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19285) Add per-table latency histograms
Date Fri, 17 Nov 2017 23:07:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16257723#comment-16257723
] 

Josh Elser commented on HBASE-19285:
------------------------------------

Well, color me surprised. Initial tests show no impact. This makes me think I did something
wrong.

{noformat}
$ bin/hbase pe --latency --nomapred --presplit=1000 --valueSize=1000 --rows=100000 sequentialWrite
30
{noformat}

{noformat}
### With table metrics

2017-11-17 22:36:53,939 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [59063, 59046, 60074, 59116, 59436, 59829, 58789, 59049, 60074, 59087, 58798,
59934, 59328, 59325, 59790, 59317, 59961, 60116, 58530, 59394, 59515, 58430, 58510, 59290,
59376, 58812, 59762, 58578, 59929, 59388]
2017-11-17 22:36:53,940 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
58430ms    Max: 60116ms    Avg: 59321ms

2017-11-17 22:40:24,731 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [56313, 56755, 56521, 52565, 54266, 53052, 56263, 54206, 56329, 51822, 52421,
54228, 53171, 56686, 56707, 53018, 50557, 56529, 56326, 56704, 56519, 54258, 56503, 54292,
56338, 53167, 56508, 53116, 52534, 56193]
2017-11-17 22:40:24,733 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
50557ms    Max: 56755ms    Avg: 54795ms

2017-11-17 22:44:45,189 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [64965, 63050, 63186, 63057, 63068, 63450, 63929, 63121, 63272, 62800, 64640,
64476, 63888, 62953, 64284, 62958, 64466, 64607, 64266, 63359, 64628, 64374, 64948, 63883,
64322, 63837, 63264, 64348, 64670, 63408]
2017-11-17 22:44:45,190 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
62800ms    Max: 64965ms    Avg: 63849ms

### Without table metrics (stock)

2017-11-17 22:52:01,694 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [68170, 68175, 61443, 69265, 69226, 69447, 69371, 60942, 69250, 69088, 65739,
65723, 65756, 65764, 63096, 68151, 65782, 63133, 68174, 68193, 63076, 61539, 65809, 68167,
68984, 69373, 67716, 68154, 65756, 69024]
2017-11-17 22:52:01,694 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
60942ms    Max: 69447ms    Avg: 66716ms

2017-11-17 22:57:35,774 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [62112, 61847, 62267, 61856, 62088, 61863, 59649, 61915, 62237, 60353, 59469,
60340, 59375, 60396, 59450, 59625, 62170, 60338, 59000, 61362, 60308, 61921, 61300, 61834,
62142, 59622, 62203, 61935, 59639, 59467]
2017-11-17 22:57:35,775 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
59000ms    Max: 62267ms    Avg: 60936ms

2017-11-17 23:00:28,712 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary
of timings (ms): [58838, 64460, 63499, 65246, 59610, 60985, 63585, 60948, 65323, 58409, 58478,
63565, 63601, 61111, 64464, 65149, 60909, 59680, 65169, 61050, 65371, 65197, 59647, 60922,
59605, 65232, 63591, 65147, 65307, 64475]
2017-11-17 23:00:28,713 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min:
58409ms    Max: 65371ms    Avg: 62619ms
{noformat}

Single RS, 8G heap a couple of memstore flush config tweaks to try to smooth out the 1K regions
on a single RS. My only guess is that I'm still getting blocked on flushes which is dominating
execution time. Let me poke with JFR or YourKit.

> Add per-table latency histograms
> --------------------------------
>
>                 Key: HBASE-19285
>                 URL: https://issues.apache.org/jira/browse/HBASE-19285
>             Project: HBase
>          Issue Type: Bug
>          Components: metrics
>            Reporter: Clay B.
>            Assignee: Josh Elser
>            Priority: Critical
>             Fix For: 2.0.0, 1.4.0, 1.3.3
>
>
> HBASE-17017 removed the per-region latency histograms (e.g. Get, Put, Scan at p75, p85,
etc)
> HBASE-15518 added some per-table metrics, but not the latency histograms.
> Given the previous conversations, it seems like it these per-table aggregations weren't
intentionally omitted, just never re-implemented after the per-region removal. They're some
really nice out-of-the-box metrics we can provide to our users/admins as long as it's not
detrimental.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message