hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6841) Meta prefetching is slower than doing multiple meta lookups
Date Thu, 20 Sep 2012 17:14:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459753#comment-13459753

Lars Hofhansl commented on HBASE-6841:

ensureZookeeperTrackers only does any work when ZK was not setup already. Previously this
was sprinkled all over, but did the same amount of work. That part is OK, I think (in 0.94
HConnection needs a working ZK connection to function).

Just checked, HConnectionManager.execute is called all over the place (12 different places
in HConnectionManager and HTable).
We do some weird stuff. For example setting the prefetching is also done through execute (i.e.
get a - potentially - new HConntionImplementation, call setPrefetching on it, then close it).

On the face of it, this does not look like it's specific to region prefetching, but the fact
that repeatedly do something with a new connection.

@J-D: Did you disable this path and found faster?
> Meta prefetching is slower than doing multiple meta lookups
> -----------------------------------------------------------
>                 Key: HBASE-6841
>                 URL: https://issues.apache.org/jira/browse/HBASE-6841
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jean-Daniel Cryans
>            Priority: Critical
>             Fix For: 0.94.2
> I got myself into a situation where I needed to truncate a massive table while it was
getting hits and surprisingly the clients were not recovering. What I see in the logs is that
every time we prefetch .META. we setup a new HConnection because we close it on the way out.
It's awfully slow.
> We should just turn it off or make it useful. jstacks coming up.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message