hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthick Sankarachary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3766) Manage Connections Through Reference Counts
Date Mon, 11 Apr 2011 21:35:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018582#comment-13018582

Karthick Sankarachary commented on HBASE-3766:

{quote}About the patch, I see you increment in HTable's constructor, but if the user is reusing
a conf object then the counter becomes wrong right?{quote}

Let's say that a certain conf object is used to create 2 HTable instances. As a result, the
reference count on the underlying connection object will be incremented twice, which is the
desired behavior. Then, when you close one of the HTables, it will decrement the reference
count on the connection, but not close it, since it is still in-use. Later, when you close
the other HTable, the reference count will drop to zero, at which point, the underlying connection
gets deleted. On the other hand, if you use different conf objects to create the HTables,
they will end up having separate reference counts. Please correct me if I'm misunderstood
the question.

{quote}Also the counter itself isn't accessed in a synchronize way (as multiple threads using
different HTables would need) so this could pose a problem.{quote}

Good catch. I revised the patch such that the reference count is accessed atomically.

> Manage Connections Through Reference Counts
> -------------------------------------------
>                 Key: HBASE-3766
>                 URL: https://issues.apache.org/jira/browse/HBASE-3766
>             Project: HBase
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 0.90.2
>            Reporter: Karthick Sankarachary
>            Assignee: Karthick Sankarachary
>            Priority: Minor
>             Fix For: 0.90.3
>         Attachments: HBASE-3766.patch, HBASE-3766.patch
>   Original Estimate: 1h
>  Remaining Estimate: 1h
> As of now, the onus is on the developer to explicitly close connections once they're
done using them. Furthermore, since connections are managed based on the identity of the {{Configuration}}
object, one is forced to clone the configuration object in order to be able to clean it up
safely (for a case in point, see {{HTablePool's}} constructor). As a matter of fact, this
issue has been well-documented in the HConnectionManager class:
> {quote}
> But sharing connections makes clean up of {{HConnection}} instances a little awkward.
 Currently, clients cleanup by calling {{#deleteConnection(Configuration, boolean)}}.  This
will shutdown the zookeeper connection the {{HConnection}} was using and clean up all {{HConnection}}
resources as well as stopping proxies to servers out on the cluster. Not running the cleanup
will not end the world; it'll just stall the closeup some and spew some zookeeper connection
failed messages into the log.  Running the cleanup on a {{HConnection}} that is subsequently
used by another will cause breakage so be careful running cleanup. To create a {{HConnection}}
that is not shared by others, you can create a new {{Configuration}} instance, pass this new
instance to {{#getConnection(Configuration)}}, and then when done, close it up by doing something
like the following:
> {code}
>  Configuration newConfig = new Configuration(originalConf);
>  HConnection connection = HConnectionManager.getConnection(newConfig);
>  // Use the connection to your hearts' delight and then when done...
>  HConnectionManager.deleteConnection(newConfig, true);
> {code}
> {quote}
> Here, we propose a reference-count based mechanism for managing connections that will
allow {{HTables}} to clean up after themselves. In particular, we extend the {{HConnectionInterface}}
interface so as to facilitate reference counting, where, typically, a reference indicates
that it is being used by a {{HTable}}, although there could be other sources. 
> To elaborate, when a HTable is constructed, it increments the reference count on the
connection given to it. Similarly, when it is closed, that reference count is decremented.
In the event there are no more references to that connection, {{HTable#close}} takes it upon
itself to delete the connection, thereby sparing the developer from doing so.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message