hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3855) Performance degradation of memstore because reseek is linear
Date Fri, 06 May 2011 06:46:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029787#comment-13029787
] 

dhruba borthakur commented on HBASE-3855:
-----------------------------------------

I have a cf with three columns. I insert 100K puts on a single rowkey with random column values.
all these kvs are cached in the memstore.  A single scan to retrieve the values for two specified
columnname takes about 4 millisecons of CPU without this patch and about 0.025 millieseconds
of cpu with this patch. Unfortunately, I cannot share the test program because it is intertwined
with quite a few non-opensource-able code.

However, I think the patch will benefit all cases because it is better to do a O(logn) lookup
rather then even a small number of sequential scans.


 

> Performance degradation of memstore because reseek is linear
> ------------------------------------------------------------
>
>                 Key: HBASE-3855
>                 URL: https://issues.apache.org/jira/browse/HBASE-3855
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.94.0
>
>         Attachments: memstoreReseek.txt
>
>
> The scanner use reseek to find the next row (or next column) as part of a scan. The reseek
code iterates over a Set to position itself at the right place. If there are many thousands
of kvs that need to be skipped over, then the time-cost is very high. In this case, a seek
would be far lesser in cost than a reseek.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message