hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12213) HFileBlock backed by Array of ByteBuffers
Date Fri, 03 Jul 2015 06:42:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612900#comment-14612900
] 

ramkrishna.s.vasudevan commented on HBASE-12213:
------------------------------------------------

Did some perf testing using the Performance evaluation tool.
For range scan there is no difference between with and without patch
ScanRange10000 rows=10000 with 25 threads, filterAll = true
without patch
{code}
2015-07-03 17:06:35,332 INFO  [main] hbase.PerformanceEvaluation: [RandomScanWithRange10000Test]
Summary of timings (ms): [573388, 574365, 573071, 571351, 572586, 572020, 572732, 573370,
573627, 573146, 573245, 574017, 573049, 574283, 571470, 574312, 574539, 574515, 571540, 574108,
573530, 574177, 574143, 572927, 574541]
2015-07-03 17:06:35,333 INFO  [main] hbase.PerformanceEvaluation: [RandomScanWithRange10000Test]
       Min: 571351ms   Max: 574541ms   Avg: 573362ms

{code}
with patch
{code}
2015-07-03 16:49:57,711 INFO  [main] hbase.PerformanceEvaluation: [RandomScanWithRange10000Test]
Summary of timings (ms): [572276, 569970, 572610, 571517, 572027, 570168, 570932, 572951,
570492, 572410, 572037, 571495, 572238, 572055, 572114, 571142, 572500, 569813, 572383, 572223,
571191, 572197, 571473, 571235, 569209]
2015-07-03 16:49:57,715 INFO  [main] hbase.PerformanceEvaluation: [RandomScanWithRange10000Test]
       Min: 569209ms   Max: 572951ms   Avg: 571546ms
{code}

But when we do gets which involves lot of blockSeek code 
randomReads = 10 threads, multiGets = 100, filterAll = true
withoutpatch
{code}
2015-07-03 17:13:37,172 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [158292, 158191, 157503, 158375, 158182, 158901, 157680, 158583, 158388,
158279]
2015-07-03 17:13:37,173 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
157503ms   Max: 158901ms   Avg: 158237ms

2015-07-03 17:33:36,854 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [163757, 163478, 163175, 163876, 163490, 163187, 163813, 163937, 162839,
163398]
2015-07-03 17:33:36,855 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
162839ms   Max: 163937ms   Avg: 163495ms

2015-07-03 17:36:44,255 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [161414, 162715, 162266, 162409, 161884, 162161, 162235, 162077, 162852,
161575]
2015-07-03 17:36:44,256 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
161414ms   Max: 162852ms   Avg: 162158ms
{code}
with patch
{code}
2015-07-03 17:26:02,496 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [150311, 150721, 150350, 149602, 150339, 150593, 151117, 149046, 150403,
150087]
2015-07-03 17:26:02,497 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
149046ms   Max: 151117ms   Avg: 150256ms

2015-07-03 17:43:52,968 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [148812, 148077, 148482, 148160, 148479, 147835, 148461, 147683, 148499,
149299]
2015-07-03 17:43:52,970 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
147683ms   Max: 149299ms   Avg: 148378ms

2015-07-03 17:46:30,247 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary
of timings (ms): [149254, 148839, 148658, 148982, 148443, 147960, 148590, 148508, 148855,
148767]
2015-07-03 17:46:30,248 INFO  [main] hbase.PerformanceEvaluation: [RandomReadTest]      Min:
147960ms   Max: 149254ms   Avg: 148685ms

{code}
We see significant gain.  Currently note that we are copying the offheap buckets to the onheap
single BB.  The blockSeek positional based reads and some optimization in the KeyOnlyKV is
the reason for this performance gain.   Even without MBB we should be able to see the same
gain because now the MBB is going to act as a wrapper over the single BB that was formed from
the underlying offheap buckets.

> HFileBlock backed by Array of ByteBuffers
> -----------------------------------------
>
>                 Key: HBASE-12213
>                 URL: https://issues.apache.org/jira/browse/HBASE-12213
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: Anoop Sam John
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-12213_1.patch
>
>
> In L2 cache (offheap) an HFile block might have been cached into multiple chunks of buffers.
If HFileBlock need single BB, we will end up in recreation of bigger BB and copying. Instead
we can make HFileBlock to serve data from an array of BBs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message