hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philippe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3962) HConnectionManager.getConnection(HBaseConfiguration) returns new connection in default HTable constructor
Date Fri, 10 Jun 2011 15:58:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047262#comment-13047262

Philippe commented on HBASE-3962:


Thank you very much for your message. I did check the code source from the latest release,
but not of the trunk actually. It seems you covered all the points I mentioned. Great !

However, I’m not really sure about a couple of things:

a) If I use the ‘same’ configuration each time I create a table, I’ll use the same connection
then. I have a server that connects to HBase. Isn’t there a performance issue if the server
will use only one connection to the quorum? As the quorum allows me to MAX_CLIENT_CNXNS connections,
should we use as many connections as possible instead of just one for the same configuration?

b) The number of cached HConnection are now related to the number of connections allowed by
the zookeeper ensemble. But if I pass different configuration, can’t they supposed to use
different zookeeper quorum (I guess that may be an extremely rare case)? If so, is there really
a point to limit the number of cached HConnections?

> HConnectionManager.getConnection(HBaseConfiguration) returns new connection in default
HTable constructor
> ---------------------------------------------------------------------------------------------------------
>                 Key: HBASE-3962
>                 URL: https://issues.apache.org/jira/browse/HBASE-3962
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.1
>            Reporter: Philippe
> The HBase instance are currently indexed by Configuration, which since HBASE-1976 does
not have any other equivalence that the object equivalence.
> So, everytime a new configuration is passed to the method a new connection is created.
> If we create many HTable connections with the same configuration, there is no problem:
> HBaseConfiguration config = HBaseConfiguration.create();
> HTable table 1 = new HTable(config, "table1"); // init connection
> HTable table 2 = new HTable(config, "table2"); // re-use connection
> HTable table 3 = new HTable(config, "table3"); // re-use connection
> However, if we call the default constructor, or re-call HBaseConfiguration.create();,
we will pass a new instance of the configuration to the constructor. This will cause many
connections to be created:
> HTable table 1 = new HTable("table1"); // init connection
> HTable table 2 = new HTable("table2"); // init new connection
> HTable table 3 = new HTable("table3"); // init new connection
> I know connection should be pooled, but sometimes we have to create a new connection,
and without having access to a previously instanced configuration object.
> Since zookeeper has a max client connection (default was 30, now is 10), after creating
30 instances of HTable, we can no longer access to the database.
> In addition to this, the HBASE_INSTANCES map does not close the connection when removing
the eldest entry. So if we have a larger maxConnection value than the hard-coded MAX_CACHED_HBASE_INSTANCES
variable, connections will remain but won't be closed. MAX_CACHED_HBASE_INSTANCES should actually
be set from the hbase.zookeeper.property.maxClientCnxns parameter (value + 1).

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message