hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-887) Fix a hotspot in scanners
Date Wed, 17 Sep 2008 21:21:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631953#action_12631953
] 

Jean-Daniel Cryans commented on HBASE-887:
------------------------------------------

Some tests I did with a modified PE. I ran the PE with a number from 10 to 3000 and incremented
by 10. The garbage collector was called after each run. Here are the results:

{quote}
08/09/17 16:48:54 INFO hbase.ManyPerformanceEvaluation: 10 42725
08/09/17 16:49:31 INFO hbase.ManyPerformanceEvaluation: 20 36744
08/09/17 16:50:06 INFO hbase.ManyPerformanceEvaluation: 30 34928
08/09/17 16:50:40 INFO hbase.ManyPerformanceEvaluation: 40 34150
08/09/17 16:51:14 INFO hbase.ManyPerformanceEvaluation: 50 33664
08/09/17 16:51:47 INFO hbase.ManyPerformanceEvaluation: 60 33570
08/09/17 16:52:21 INFO hbase.ManyPerformanceEvaluation: 70 33142
08/09/17 16:52:54 INFO hbase.ManyPerformanceEvaluation: 80 32901
08/09/17 16:53:26 INFO hbase.ManyPerformanceEvaluation: 90 32864
08/09/17 16:53:59 INFO hbase.ManyPerformanceEvaluation: 100 32776
08/09/17 16:54:32 INFO hbase.ManyPerformanceEvaluation: 110 32725
08/09/17 16:55:05 INFO hbase.ManyPerformanceEvaluation: 120 32747
08/09/17 16:55:38 INFO hbase.ManyPerformanceEvaluation: 130 32698
08/09/17 16:56:10 INFO hbase.ManyPerformanceEvaluation: 140 32693
08/09/17 16:56:43 INFO hbase.ManyPerformanceEvaluation: 150 32643
08/09/17 16:57:16 INFO hbase.ManyPerformanceEvaluation: 160 32686
08/09/17 16:57:48 INFO hbase.ManyPerformanceEvaluation: 170 32665
08/09/17 16:58:21 INFO hbase.ManyPerformanceEvaluation: 180 32448
08/09/17 16:58:53 INFO hbase.ManyPerformanceEvaluation: 190 32004
08/09/17 16:59:25 INFO hbase.ManyPerformanceEvaluation: 200 31985
08/09/17 16:59:57 INFO hbase.ManyPerformanceEvaluation: 210 31964
08/09/17 17:00:29 INFO hbase.ManyPerformanceEvaluation: 220 31983
08/09/17 17:01:01 INFO hbase.ManyPerformanceEvaluation: 230 31778
08/09/17 17:01:32 INFO hbase.ManyPerformanceEvaluation: 240 31766
08/09/17 17:02:04 INFO hbase.ManyPerformanceEvaluation: 250 31886
08/09/17 17:02:36 INFO hbase.ManyPerformanceEvaluation: 260 31773
08/09/17 17:03:08 INFO hbase.ManyPerformanceEvaluation: 270 31709
08/09/17 17:03:40 INFO hbase.ManyPerformanceEvaluation: 280 31669
08/09/17 17:04:11 INFO hbase.ManyPerformanceEvaluation: 290 31647
08/09/17 17:04:43 INFO hbase.ManyPerformanceEvaluation: 300 31634
08/09/17 17:05:15 INFO hbase.ManyPerformanceEvaluation: 310 31674
08/09/17 17:05:47 INFO hbase.ManyPerformanceEvaluation: 320 31935
08/09/17 17:06:18 INFO hbase.ManyPerformanceEvaluation: 330 31802
08/09/17 17:06:50 INFO hbase.ManyPerformanceEvaluation: 340 31571
08/09/17 17:07:21 INFO hbase.ManyPerformanceEvaluation: 350 31484
08/09/17 17:07:53 INFO hbase.ManyPerformanceEvaluation: 360 31713
08/09/17 17:08:25 INFO hbase.ManyPerformanceEvaluation: 370 31512
08/09/17 17:08:56 INFO hbase.ManyPerformanceEvaluation: 380 31456
08/09/17 17:09:28 INFO hbase.ManyPerformanceEvaluation: 390 32055
08/09/17 17:10:01 INFO hbase.ManyPerformanceEvaluation: 400 32260
08/09/17 17:10:33 INFO hbase.ManyPerformanceEvaluation: 410 32167
08/09/17 17:11:05 INFO hbase.ManyPerformanceEvaluation: 420 31962
08/09/17 17:11:37 INFO hbase.ManyPerformanceEvaluation: 430 32003
08/09/17 17:12:09 INFO hbase.ManyPerformanceEvaluation: 440 32054
08/09/17 17:12:41 INFO hbase.ManyPerformanceEvaluation: 450 32105
08/09/17 17:13:13 INFO hbase.ManyPerformanceEvaluation: 460 32126
08/09/17 17:13:45 INFO hbase.ManyPerformanceEvaluation: 470 31961
08/09/17 17:14:17 INFO hbase.ManyPerformanceEvaluation: 480 31968
...
{quote}

It seems that 2000 was an overkill. The gain is impressive between 1 and 10 then around 300
we pretty much reduced the RPC call at it's smallest value.

> Fix a hotspot in scanners
> -------------------------
>
>                 Key: HBASE-887
>                 URL: https://issues.apache.org/jira/browse/HBASE-887
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, regionserver
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.19.0
>
>         Attachments: hbase-887-v1.patch
>
>
> When scanning, we do a lot of RPCs and this has a huge performance hit. I propose that
we add a way to fetch more rows during next() and put them in cache. This should be configurable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message