hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9751) Excessive readpoints checks in MemStoreScanner and StoreFileScanner
Date Sat, 12 Oct 2013 05:13:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793269#comment-13793269

Lars Hofhansl commented on HBASE-9751:

And note that in a large system most KVs are in HFiles that have been compacted and the probability
is high that they were older than the oldest scanner at compaction time; hence their memstoreTS
is set to 0 in the HFile, and hence this optimization works.

[~vrodionov], if you have any ideas about the areas you mention, as well as how avoid getting
the readpoint on each call to next in MemstoreScanner, I'd love to hear them.

> Excessive  readpoints checks in MemStoreScanner and StoreFileScanner
> --------------------------------------------------------------------
>                 Key: HBASE-9751
>                 URL: https://issues.apache.org/jira/browse/HBASE-9751
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.0, 0.94.12, 0.96.0
>            Reporter: Vladimir Rodionov
>            Assignee: Lars Hofhansl
>         Attachments: 9751-0.94.txt
> It seems that usage of skipKVsNewerThanReadpoint in StoreFileScanner can be greatly reduced
or even eliminated all together (HFiles are immutable and no new KVs can be inserted after
scanner instance is created). The same is true for MemStoreScanner which checks readpoint
on every next() and seek(). Each readpoint check is ThreadLocal.get() and it is quite expensive.

This message was sent by Atlassian JIRA

View raw message