hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Bautin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-4465) Lazy-seek optimization for StoreFile scanners
Date Fri, 23 Sep 2011 03:11:27 GMT
Lazy-seek optimization for StoreFile scanners

                 Key: HBASE-4465
                 URL: https://issues.apache.org/jira/browse/HBASE-4465
             Project: HBase
          Issue Type: Improvement
            Reporter: Mikhail Bautin
            Assignee: Mikhail Bautin
             Fix For: 0.92.0, 0.94.0, 0.89.20100924

Previously, if we had several StoreFiles for a column family in a region, we would seek in
each of them and only then merge the results, even though the row/column we are looking for
might only be in the most recent (and the smallest) file. Now we prioritize our reads from
those files so that we check the most recent file first. This is done by doing a "lazy seek"
which pretends that the next value in the StoreFile is (seekRow, seekColumn, lastTimestampInStoreFile),
which is earlier in the KV order than anything that might actually occur in the file. So if
we don't find the result in earlier files, that fake KV will bubble up to the top of the KV
heap and a real seek will be done. This is expected to significantly reduce the amount of
disk IO (as of 09/22/2011 we are doing dark launch testing and measurement).

This is joint work with Liyin Tang -- huge thanks to him for many helpful discussions on this
and the idea of putting fake KVs with the highest timestamp of the StoreFile in the scanner
priority queue.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message