hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Muthukkaruppan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-4443) optimize/avoid seeking to "previous" block when key you are interested in is the first one of a block
Date Mon, 19 Sep 2011 21:19:10 GMT
optimize/avoid seeking to "previous" block when key you are interested in is the first one
of a block

                 Key: HBASE-4443
                 URL: https://issues.apache.org/jira/browse/HBASE-4443
             Project: HBase
          Issue Type: Improvement
            Reporter: Kannan Muthukkaruppan

This issue primarily affects cases when you are storing large blobs, i.e. when blocks contain
small number of keys, and the chances of the key you are looking for being the first block
of a key is higher.

Say, you are looking for "row/col", and "row/col/ts=5" is the latest version of the key in
the HFile and is at the beginning of block X.

The search for the key is done by looking for "row/col/TS=Long.MAX_VAL", but this will land
us in block X-1 (because ts=Long.MAX_VAL sorts ahead of ts=5); only to find that there is
no matching "row/col" in block X-1, and then we'll advance to block X to return the value.

Seems like we should be able to optimize this somehow.

Some possibilities:
1) Suppose we track that the  file contains no deletes, and if the CF setting has MAX_VERSIONS=1,
we can know for sure that block X - 1 does not contain any relevant data, and directly position
the seek to block X. [This will also require the memstore flusher to remove extra versions
if MAX_VERSION=1 and not allow the file to contain duplicate entries for the same ROW/COL.]
 Tracking deletes will also avoid in many cases, the seek to the top of the row to look for

2) Have a dense index (1 entry per KV in the index; this might be ok for large object case
since index vs. data ratio will still be low).

3) Have the index contain the last KV of each block also in addition to the first KV. This
doubles the size of the index though.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message