hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6874) Implement prefetching for scanners
Date Thu, 01 Nov 2012 04:27:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488448#comment-13488448
] 

Lars Hofhansl commented on HBASE-6874:
--------------------------------------

[~karthik.ranga] Do you have a patch. We were just discussing something similar here and I
was about to open a similar issue before I found this one. This is even more useful with scanner
caching.

One could even go a step further and parallelize the prefetching into N threads (useful if
the results are heavily prefiltered at the server).

We do our own parallel scanner fetching (not necessarily on region or buffer boundaries),
but it would be nice if that could be generalized and be part of HBase.

                
> Implement prefetching for scanners
> ----------------------------------
>
>                 Key: HBASE-6874
>                 URL: https://issues.apache.org/jira/browse/HBASE-6874
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Karthik Ranganathan
>            Assignee: Karthik Ranganathan
>
> I did some quick experiments by scanning data that should be completely in memory and
found that adding pre-fetching increases the throughput by about 50% from 26MB/s to 39MB/s.
> The idea is to perform the next in a background thread, and keep the result ready. When
the scanner's next comes in, return the pre-computed result and issue another background read.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message