hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6841) Meta prefetching is slower than doing multiple meta lookups
Date Thu, 20 Sep 2012 20:15:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459910#comment-13459910

Lars Hofhansl commented on HBASE-6841:

I'd be OK with that. I'd also be worried that this is just a symptom. HConnectionManager.getConnection(...)
and HConnection.close() should just do some reference counting rather than actually creating/destroying
connection; which means sometime we're coming in there with a new Configuration every time...?
And even that should be handled by the Configuration equivalence code we're using now.
So if, in this case, we'd remove prefetching, we'd still have the expensive Connection setup
every time.

Then again and just to state the obvious, the prefetching is only useful for long lived connections
and then only if these connections actually use a larg'ish portion of the prefetched entries
(otherwise we're doing a lot of unnecessary work cache, and wasting memory).

Let's just disable it by default. I guess we'd do that by reversing the meaning (and name)
of regionCachePrefetchDisabledTables). Happy to make a patch if you folks agree.

> Meta prefetching is slower than doing multiple meta lookups
> -----------------------------------------------------------
>                 Key: HBASE-6841
>                 URL: https://issues.apache.org/jira/browse/HBASE-6841
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jean-Daniel Cryans
>            Priority: Critical
>             Fix For: 0.94.2
> I got myself into a situation where I needed to truncate a massive table while it was
getting hits and surprisingly the clients were not recovering. What I see in the logs is that
every time we prefetch .META. we setup a new HConnection because we close it on the way out.
It's awfully slow.
> We should just turn it off or make it useful. jstacks coming up.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message