hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xufeng (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4773) HBaseAdmin leaks ZooKeeper connections
Date Fri, 25 Nov 2011 08:22:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157024#comment-13157024
] 

xufeng commented on HBASE-4773:
-------------------------------

yes, I have tested it in my cluster.

Here is my client test code:
{noformat}
.....
  static void initHBase() throws ZooKeeperConnectionException
  {
    HBaseAdmin hbaseAdmin = null;
    Configuration config = HBaseConfiguration.create();
    config.set("hbase.zookeeper.quorum", "158.1.130.31,158.1.130.32,158.1.130.33");
    config.set("hbase.zookeeper.property.clientPort", "2181");
    
    try {
      hbaseAdmin = new HBaseAdmin(config);
      System.out.println("init sucess!");
    } catch (MasterNotRunningException e) {
      e.printStackTrace();
      initHBase();
      
    } catch (ZooKeeperConnectionException e) {
      e.printStackTrace();
      initHBase();
    }
  }
}
.....
{noformat}

In my cluster I did not start HBase process.

Run test,result of the lsof commondline is:
{noformat}
java      16735       root   72w      REG              253,3   890569     524379 /opt/xf/hadoop.log
java      16735       root   73w      REG              253,3   274338     524376 /opt/xf/HA_hadoop.log
java      16735       root   74r     FIFO                0,8      0t0  110645029 pipe
java      16735       root   75w     FIFO                0,8      0t0  110645029 pipe
java      16735       root   76u     0000                0,9        0         21 anon_inode
java      16735       root   77u     IPv6          110645030      0t0        TCP C3S31:35186->C3S33:eforward
(ESTABLISHED)
java      16735       root   78u     unix 0xffff8800cba90380      0t0  110645035 socket
java      16735       root   79u     sock                0,6      0t0  110645032 can't identify
protocol
java      16735       root   80r     FIFO                0,8      0t0  110645037 pipe
java      16735       root   81w     FIFO                0,8      0t0  110645037 pipe
java      16735       root   82u     0000                0,9        0         21 anon_inode
java      16735       root   83u     IPv6          110645038      0t0        TCP C3S31:53727->C3S31:eforward
(ESTABLISHED)
java      16735       root   84r     FIFO                0,8      0t0  110645043 pipe
java      16735       root   85w     FIFO                0,8      0t0  110645043 pipe
java      16735       root   86u     0000                0,9        0         21 anon_inode
java      16735       root   87u     IPv6          110645044      0t0        TCP C3S31:53728->C3S31:eforward
(ESTABLISHED)
java      16735       root   88r     FIFO                0,8      0t0  110645047 pipe
java      16735       root   89w     FIFO                0,8      0t0  110645047 pipe
java      16735       root   90u     0000                0,9        0         21 anon_inode
java      16735       root   91u     IPv6          110645048      0t0        TCP C3S31:47183->C3S32:eforward
(ESTABLISHED)
java      16735       root   92r     FIFO                0,8      0t0  110645050 pipe
java      16735       root   93w     FIFO                0,8      0t0  110645050 pipe
java      16735       root   94u     0000                0,9        0         21 anon_inode
java      16735       root   95u     IPv6          110645051      0t0        TCP C3S31:53730->C3S31:eforward
(ESTABLISHED)
java      16735       root   96r     FIFO                0,8      0t0  110645135 pipe
java      16735       root   97w     FIFO                0,8      0t0  110645135 pipe
java      16735       root   98u     0000                0,9        0         21 anon_inode
java      16735       root   99u     IPv6          110645136      0t0        TCP C3S31:49799->C3S31:eforward
(ESTABLISHED)
java      16735       root  100r     FIFO                0,8      0t0  110645143 pipe
java      16735       root  101w     FIFO                0,8      0t0  110645143 pipe
java      16735       root  102u     0000                0,9        0         21 anon_inode
java      16735       root  103u     IPv6          110645144      0t0        TCP C3S31:38931->C3S32:eforward
(ESTABLISHED)
java      16735       root  104r     FIFO                0,8      0t0  110645148 pipe
java      16735       root  105w     FIFO                0,8      0t0  110645148 pipe
java      16735       root  106u     0000                0,9        0         21 anon_inode
java      16735       root  107u     IPv6          110645149      0t0        TCP C3S31:59939->C3S33:eforward
(ESTABLISHED)
java      16735       root  108r     FIFO                0,8      0t0  110645507 pipe
java      16735       root  109w     FIFO                0,8      0t0  110645507 pipe
java      16735       root  110u     0000                0,9        0         21 anon_inode
java      16735       root  111u     IPv6          110645508      0t0        TCP C3S31:59940->C3S33:eforward
(ESTABLISHED)
{noformat}

The [eforward] is port of zookeeper. 

It made the connection leak because did not delete connection between client and zookeeper
when MasterNotRunningException happened.

And I also tested my patch,the result of it is :
{noformat}
java      16652       root   71r      REG              253,3   936397     524302 /opt/xf/lib/guava-r06.jar
java      16652       root   72w      REG              253,3   786418     524379 /opt/xf/hadoop.log
java      16652       root   73w      REG              253,3   262352     524376 /opt/xf/HA_hadoop.log
java      16652       root   74r     FIFO                0,8      0t0  110644817 pipe
java      16652       root   75w     FIFO                0,8      0t0  110644817 pipe
java      16652       root   76u     0000                0,9        0         21 anon_inode
java      16652       root   77u     IPv6          110644818      0t0        TCP C3S31:53993->C3S33:eforward
(ESTABLISHED)
java      16652       root   78u     unix 0xffff8800cbb1d9c0      0t0  110644491 socket
java      16652       root   79u     sock                0,6      0t0  110644488 can't identify
protocol
{noformat}
                
> HBaseAdmin leaks ZooKeeper connections
> --------------------------------------
>
>                 Key: HBASE-4773
>                 URL: https://issues.apache.org/jira/browse/HBASE-4773
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.4
>            Reporter: gaojinchao
>            Priority: Critical
>             Fix For: 0.90.5
>
>         Attachments: 4773.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
>  public HBaseAdmin(Configuration c)
>   throws MasterNotRunningException, ZooKeeperConnectionException {
>     this.conf = HBaseConfiguration.create(c);
>     this.connection = HConnectionManager.getConnection(this.conf);
>     this.pause = this.conf.getLong("hbase.client.pause", 1000);
>     this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
>     this.retryLongerMultiplier = this.conf.getInt("hbase.client.retries.longer.multiplier",
10);
>     //we should add this code and close the zk connection
>     try{
>       this.connection.getMaster();
>     }catch(MasterNotRunningException e){
>       HConnectionManager.deleteConnection(conf, false);
>       throw e;      
>     }
>   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message