hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: Registered listeners not getting removed - Leading to memory leak
Date Tue, 20 Dec 2011 18:33:49 GMT
Will that lead to more Zookeeper server churn?



________________________________
 From: Ramkrishna S Vasudevan <ramkrishna.vasudevan@huawei.com>
To: 'Ted Yu' <yuzhihong@gmail.com>; dev@hbase.apache.org 
Cc: 'lars hofhansl' <lhofhansl@yahoo.com> 
Sent: Monday, December 19, 2011 9:54 PM
Subject: RE: Registered listeners not getting removed - Leading to memory leak
 

 
Hi 
 
I just analysed.. I think every time we
can create a new zk watcher and close them in CatalogTracker stop() as how it
is done in trunk.
 
Regards
Ram

________________________________
 
From:Ted Yu
[mailto:yuzhihong@gmail.com] 
Sent: Tuesday, December 20, 2011
9:32 AM
To: dev@hbase.apache.org;
ramkrishna.vasudevan@huawei.com
Cc: lars hofhansl
Subject: Re: Registered listeners
not getting removed - Leading to memory leak
 
How about creating new
zookeeper watcher in 0.90, following TRUNK's example ?

Cheers
On Mon, Dec 19, 2011 at 5:32 PM, Ramkrishna S Vasudevan <ramkrishna.vasudevan@huawei.com>
wrote:
Hi All



In trunk the problem doesnot exist.  Every time when we do
getCatalogTracker
as part of the constructor



atalogTracker(final ZooKeeperWatcher zk, final Configuration conf,
   
 HConnection connection, Abortable abortable, final int defaultTimeout)

 throws IOException {
   this.connection = connection;
   if (abortable == null) {
     // A connection is abortable.
     this.abortable = this.connection;
   }
   Abortable throwableAborter = new Abortable() {

     @Override
     public void abort(String why, Throwable e) {
       throw new RuntimeException(why, e);
     }

     @Override
     public boolean isAborted() {
       return true;
     }

   };
   if (zk == null) {
     // Create our own.  Set flag so we tear it down on
stop.
     this.zookeeper =
       new ZooKeeperWatcher(conf,
"catalogtracker-on-" +
connection.toString(),
         abortable);
     instantiatedzkw = true;
   } else {
     this.zookeeper = zk;

Everytime as zk is null we create new zookeeper watcher..  But in the
0.90.x
version we don't do that.. we get the zkwatcher from the connection object.
That is why the listener is not getting cleared.

Any suggestions on this?



Regards

Ram

 _____

From: lars hofhansl [mailto:lhofhansl@yahoo.com]
Sent: Tuesday, December 20, 2011 12:17 AM
To: dev@hbase.apache.org; ramkrishna.vasudevan@huawei.com
Subject: Re: Registered listeners not getting removed - Leading to memory
leak



A friend of mine who is evaluating HBase just told that calling
HBaseAdmin.tableExists frequently slows down the client.

This is probably the same issue. He was using stock CHD3 HBase.



-- Lars



 _____
From: Ramkrishna S
Vasudevan <ramkrishna.vasudevan@huawei.com>
To: dev@hbase.apache.org
Sent: Monday, December 19, 2011 3:59 AM
Subject: Registered listeners not getting removed - Leading to memory leak

Hi



From the HBaseAdmin if i try to use apis like tableExists, compact, flush,
closeRegion all creates new CataLogTracker.  Everytime we register a
listener but we dont

remove the listeners.

Note that, every catalog tracker creation calls HConnection.getConnection().

On continuous run of admin apis this is leading to memory leak.  Correct
me
if am wrong.



Any suggestions on this on how to clear the listeners?



Regards

Ram
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message