hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
Date Mon, 13 May 2013 23:23:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13656515#comment-13656515

Jimmy Xiang commented on HBASE-8420:

bq.    I think you have a leak if the client doesn't come back to get the prefetched data,
at least you should decrement globalPrefetchedResultSize in RegionScannerHolder.closeScanner().

Good catch.  I will fix this.

bq.    512MB seems big, I'd be more comfortable with 128MB.

How about 256 MB?  128MB is a little small.

bq.    Is it possible to unit test this?

For the memory cap, it's probably hard to unit test. I am doing some manual testing.

bq.    Have you run a concurrent workload against capped pre-fetching?

Will set the cap to a small number and do some manual testing.

bq.    Should we report on the state of globalPrefetchedResultSize? I'm wondering what could
be helpful to see when debugging scan slowness WRT this rate limiting. Maybe a TRACE every
time we skip a prefetch and output totalPrefetchedResultSize?

Sure.  Will do.

> Port  HBASE-6874  Implement prefetching for scanners from 0.89-fb
> -----------------------------------------------------------------
>                 Key: HBASE-8420
>                 URL: https://issues.apache.org/jira/browse/HBASE-8420
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jimmy Xiang
>            Assignee: Jimmy Xiang
>         Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch, trunk-8420_v1.patch, trunk-8420_v2.patch,
> This should help scanner performance.  We should have it in trunk.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message