hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization for Get Operations
Date Thu, 01 Jun 2017 21:11:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033707#comment-16033707

Eshcar Hillel commented on HBASE-17339:

After some time away from this Jira, and some additional experiments and digging into the
code, here are our current understanding:
HBase already implements some optimization which makes the current suggestion less critical.
I will try to explain it in a nutshell.
As mentioned, a get operation is divided into two main steps
(1) creating and filtering all HFile scanners and memory scanners,
(2) applying the next operation which retrieves the result for the operation.
HBase defers the seek operation of the scanners as much as possible. In step (1) all scanners
are combined in a key-value heap which is sorted by the top key of all scanners. However if
there is more than one scanner, then the HFiles scanners do not apply real seek. Instead they
set the current cell to be a fake cell which simulates as if a seek to the key was done. 
In cases were the key can be found both in memory and on disk memory segments have higher
timestamps, and they reside at the top of the heap. Finally, in step (2) the store scanner
gets the result from the scanners heap. It starts querying the scanners at the top. Only at
this point if an HFile scanner is polled from the heap and no real seek was done HBase seeks
the key in the file. This seek might end up finding the blocks in the cache or it retrieves
them from disk.
In addition, in step (1) filtering HFile scanners requires reading HFile metadata and bloom
filters -- in most cases these can be found in cache.

The optimization implemented in this Jira takes a different approach by trying to only look
in memory segments as first step. When the data is found in memory this indeed reduces latency
since it avoids the need to read HFile metadata and bloom filters and manages a bigger scanners
heap, but when the data is only on disk it incurs the overhead of scanning the data twice
(memory only and then full scan).

The question is, given this understanding is there a point in having the new optimization,
or are we satisfied with the current one?
Is there a known scenario where not all bloom filters and metadata blocks are found in the

> Scan-Memory-First Optimization for Get Operations
> -------------------------------------------------
>                 Key: HBASE-17339
>                 URL: https://issues.apache.org/jira/browse/HBASE-17339
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-17339-V01.patch, HBASE-17339-V02.patch, HBASE-17339-V03.patch,
HBASE-17339-V03.patch, HBASE-17339-V04.patch, HBASE-17339-V05.patch, HBASE-17339-V06.patch,
> The current implementation of a get operation (to retrieve values for a specific key)
scans through all relevant stores of the region; for each store both memory components (memstores
segments) and disk components (hfiles) are scanned in parallel.
> We suggest to apply an optimization that speculatively scans memory-only components first
and only if the result is incomplete scans both memory and disk.

This message was sent by Atlassian JIRA

View raw message