accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <>
Subject Re: PermGen leak in JBoss
Date Wed, 24 Apr 2013 16:22:59 GMT
On Wed, Apr 24, 2013 at 11:21 AM, Heath Abelson

> Hi, I am a colleague of Michael's, I can provide more info
> The following are the 5 threads that are not being cleaned up on an
> undeploy. It's hard to get more stack info than this.
> Daemon Thread [FileWatchdog] (Running)
> Daemon Thread [MSC service thread 1-5-SendThread(cbdbtestbox:2181)]
> (Running)
> Daemon Thread [MSC service thread 1-5-EventThread] (Running)
> Daemon Thread [Thrift Connection Pool Checker] (Running)
> Daemon Thread [FileWatchdog] (Running)
> Using Plumbr, the report states that the SendThread and EventThread above
> both contain an instance of org.apache.zookeeper.ClientCnxn

I tried this on 1.4 and then ran jstack using following program.

public class Test {
  public static void main(String[] args) throws Exception {
    ZooKeeperInstance instance = new ZooKeeperInstance("test14",
    Connector connector = instance.getConnector("root", "secret");



I see the following threads after filtering out java threads and the thread
calling main.  I am running this code in a different way.  I used the
accumulo script to execute it, which uses Accumulo start, which introduces
the AccumuloFilesystemAlterationMonitor thread.  I am seeing three of the
threads you see, I am not seeing the FileWatchDog threads.

"Thrift Connection Pool Checker" daemon prio=10 tid=0x00007f861839f800
nid=0x61f0 waiting on condition [0x00007f8652a62000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)

"Test-EventThread" daemon prio=10 tid=0x00007f8618119000 nid=0x61ee waiting
on condition [0x00007f8652d02000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000bb402270> (a
at java.util.concurrent.locks.LockSupport.park(
at org.apache.zookeeper.ClientCnxn$

"Test-SendThread(localhost:2181)" daemon prio=10 tid=0x00007f8618103000
nid=0x61ed runnable [0x00007f8652e03000]
   java.lang.Thread.State: RUNNABLE
at Method)
- locked <0x00000000bb3e0e88> (a$2)
- locked <0x00000000bb3e0e78> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000bb3e0aa8> (a
at org.apache.zookeeper.ClientCnxn$

daemon prio=10 tid=0x00007f86a4387000 nid=0x61e9 waiting on condition
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)

Should probably open a ticket about this issue.  We could probably do
something about the Thrift Connection Pool Checker threads.   I am not sure
if the zookeeper threads are an issue w/ zookeeper or an issue w/ how
accumulo is using zookeeper.

> -Heath
> -----Original Message-----
> From: Michael Giordano
> Sent: Wednesday, April 24, 2013 9:48 AM
> To:
> Subject: PermGen leak in JBoss
> Under version 1.3.7 we are using the following code to initialize a
> cloudbase connection during initialization of our web app:
>                         ZooKeeperInstance instance = new
> ZooKeeperInstance(instanceName, zooKeepers);
>                         connector = instance.getConnector(userId,
> password.getBytes());
> The problem is that under the hood, this call creates several (5) threads
> that are not cleaned up when the app is undeployed in JBoss. This is
> occurring without performing any scans or interacting with cloudbase in any
> other way. After relatively few redeploys of the app, the PermGen Space is
> OOM.
> I can't find any reference in the cloudbase API akin to a close() method
> for the Connector object. This is a classloader leak effecting any webapp
> that is accessing cloudbase directly. The result of this leak is not simply
> orphaned threads, but thousands of classes not gc'd because the classloader
> itself can't be gc'd. This is what is filling up PermGen.
> Has anyone discovered this particular issue? Has anyone discovered a
> solution?
> Thanks,
> Mike G.
> NetCentric Technology, Inc.
> 3349 Route 138, Building A
> Wall, NJ  07719
> Phone: 732-544-0888

View raw message