hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-2027) HConnectionManager.HBASE_INSTANCES leaks TableServers
Date Mon, 07 Dec 2009 17:49:18 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dave Latham updated HBASE-2027:

    Attachment: 2027-LRU.patch

Updated patch to set the max cached HConnection instances to 31.

I gave it a bit more thought, and I'm not sure what the best way would look like to track
and support different connection sets is, so I propose we go with this patch as a definite
improvement over the current set up and leave further improvement to future work.

> HConnectionManager.HBASE_INSTANCES leaks TableServers
> -----------------------------------------------------
>                 Key: HBASE-2027
>                 URL: https://issues.apache.org/jira/browse/HBASE-2027
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Dave Latham
>            Assignee: Dave Latham
>             Fix For: 0.20.3, 0.21.0
>         Attachments: 2027-LRU.patch
> HConnectionManager.HBASE_INSTANCES is a WeakHashMap from HBaseConfiguration to TableServers.
 However, each TableServers has a strong reference back to the HBaseConfiguration key so they
are never freed.  (See note at http://java.sun.com/javase/6/docs/api/java/util/WeakHashMap.html
: "Implementation note: The value objects in a WeakHashMap are held by ordinary strong references.
Thus care should be taken to ensure that value objects do not strongly refer to their own
keys, either directly or indirectly, since that will prevent the keys from being discarded.")
> Moreover, HBaseConfiguration implements hashCode() but not equals() so identical HBaseConfiguration
objects each get their own TableServers object.
> We had a long running HBase client process that was creating new HTable() objects, each
creating a new HBaseConfiguration() and thus a new TableServers object.  It eventually went
OOM, and gave a heap dump indicating 360 MB of data retained by HBASE_INSTANCES.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message