hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13291) Lift the scan ceiling
Date Wed, 01 Apr 2015 06:46:54 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390088#comment-14390088
] 

stack commented on HBASE-13291:
-------------------------------

bq. I think when we discussed on this API during tags it was decided that it would be redundant
as getTagsLength() would be enough.

Yeah, we probably said that.

I have rig w/ CPU maxed out and am trying to knock off the top consumers so we can get more
out when a hot box. In this pure scan scenario, all from blockcache, the repeated parse of
Cell lengths -- key, value, row, qualifier -- when we want to check pieces of it (row, family,
etc.) adds up.

We could cache some of these calculations (as you do in your BB patch) but while back we thought
it not worth the bulk-up in the heap. I could retry it...

> Lift the scan ceiling
> ---------------------
>
>                 Key: HBASE-13291
>                 URL: https://issues.apache.org/jira/browse/HBASE-13291
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 1.0.0
>            Reporter: stack
>            Assignee: stack
>         Attachments: 13291.hacks.txt, 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13
PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt,
q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg
>
>
> Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling'
properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs
 when 4 clients each with 10 threads doing scan 1000 rows.  If I add '--filterAll' argument
(do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops
a second.
> Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark
art going on. Let me try figure it... Filing issue in meantime to keep score in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message