hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2925) LRU of HConnectionManager.HBASE_INSTANCES breaks if HBaseConfiguration is changed
Date Fri, 03 Sep 2010 18:27:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906002#action_12906002

stack commented on HBASE-2925:

@Robert ...I wonder if it defeats the purpose of having a connection cache from the first

As I see it, the main benefit to the cache is saving on region lookups, setup of zk connection,
and master proxy setup.  Having the likes of the following config cached is secondary: e.g.

    private final long pause;
    private final int numRetries;
    private final int maxRPCAttempts;
    private final long rpcTimeout;
    private final int prefetchRegionLimit;

In fact, I'm now thinking that if a user changes any of the above in a Configuration that
is being used as a key in HCM#HBASE_INSTANCES, that its ok if we don't notice the change as
long as we doc the fact that the Configuration is read on instantiation of the HTable and
then no more.  I don't tthink this should 'surprise' the user too much and they can just go
create new HTable with the new Configuration if they really want their new config. to take
hold (Making things work this way is similar to what you suggest only you would copy the Configuration
before adding it as a key).

As to your last suggestion, I think we should move away from trying to equate Configurations
at all; there be daemons that way.

Let me know what you think.  If you are agreeable, I'll work up the patch some more mostly
adding doc clarifying what we've agreed here.

> LRU of HConnectionManager.HBASE_INSTANCES breaks if HBaseConfiguration is changed
> ---------------------------------------------------------------------------------
>                 Key: HBASE-2925
>                 URL: https://issues.apache.org/jira/browse/HBASE-2925
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3, 0.90.0
>            Reporter: Robert Mahfoud
>         Attachments: 2925.txt, SimpleHConnectionManagerLeakReplicator.java
> {{HConnectionManager.getConnection(config)}} caches the created {{TableServer}} in {{HBASE_INSTANCES}}
(a {{LinkedHashMap}} ) which is keyed by the configuration instance itself.
> Given the current implementation of {{hashCode()}} (and {{equals()}}) of {{HBaseConfiguration}},
the hash code of the configuration is changed if any of its properties are changed, which
will cause the keys of {{HBASE_INSTANCES}} to be inconsistent with the hashtable that contains
them, making some entries unreachable.
> In this case, when the map's LRU strategy needs to remove the oldest entry, it tries
to remove it based on the oldest key, which no longer gives the original hash code, therefore
the lookup in {{HBASE_INSTANCES.remove(oldest)}} doesn't actually remove anything.
> This has been observed to lead to OOM errors in long running clients.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message