manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erlend GarĂ¥sen <e.f.gara...@usit.uio.no>
Subject Re: max_pred_locks_per_transaction
Date Tue, 05 Feb 2013 12:38:34 GMT
On 04.02.13 14.40, Karl Wright wrote:
> There's code now committed to trunk which might help diagnose this
> problem.  It's part of CONNECTORS-638 and is meant to find database
> connection handle leaks.

Thanks! I will deploy trunk on our test server and start a new crawl 
later today.

BTW, here's the thread dump from the Agent - generaed by usint kill 
-QUIT <pid>:

"Thread-5386784" daemon prio=10 tid=0xdbb67000 nid=0x318f runnable 
[0xdc6fe000]
    java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
	at 
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
	- locked <0xe41eeea0> (a java.lang.Object)
	at 
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820)
	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
	- locked <0xe41f0a48> (a com.sun.net.ssl.internal.ssl.AppInputStream)
	at 
org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:145)
	at 
org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:114)
	at 
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)
	at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:274)
	at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1660)
	at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
	- locked <0xe41efe00> (a org.postgresql.core.v3.QueryExecutorImpl)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:273)
	at org.apache.manifoldcf.core.database.Database.execute(Database.java:826)
	at 
org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Database.java:641)

"Thread-5386763" daemon prio=10 tid=0xd8202800 nid=0x3171 runnable 
[0xdc369000]
    java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
	at 
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
	- locked <0xe41dbac8> (a java.lang.Object)
	at 
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820)
	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
	- locked <0xe41dcd78> (a com.sun.net.ssl.internal.ssl.AppInputStream)
	at 
org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:145)
	at 
org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:114)
	at 
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)
	at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:274)
	at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1660)
	at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
	- locked <0xe41dd280> (a org.postgresql.core.v3.QueryExecutorImpl)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
	at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:334)
	at org.apache.manifoldcf.core.database.Database.execute(Database.java:840)
	at 
org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Database.java:641)

"MultiThreadedHttpConnectionManager cleanup" daemon prio=10 
tid=0xddc92800 nid=0x1ca in Object.wait() [0xdccad000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
	- locked <0xe418ff40> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
	at 
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(Unknown 
Source)

"Idle cleanup thread" daemon prio=10 tid=0xde729800 nid=0x1f56 in 
Object.wait() [0xdd3e8000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:919)
	- locked <0xf272fe90> (a java.lang.Integer)
	at 
org.apache.manifoldcf.crawler.system.IdleCleanupThread.run(IdleCleanupThread.java:70)

"Seeding thread" daemon prio=10 tid=0xde728000 nid=0x1f55 in 
Object.wait() [0xdcfcb000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.getJobsReadyForSeeding(JobManager.java:5706)
	at 
org.apache.manifoldcf.crawler.system.SeedingThread.run(SeedingThread.java:94)

"Job reset thread" daemon prio=10 tid=0xde726800 nid=0x1f53 in 
Object.wait() [0xdd06d000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.finishJobStops(JobManager.java:6673)
	at 
org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:68)

"Document delete thread '9'" daemon prio=10 tid=0xde725000 nid=0x1f52 in 
Object.wait() [0xdd0be000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '8'" daemon prio=10 tid=0xde724000 nid=0x1f51 in 
Object.wait() [0xdd2f5000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '7'" daemon prio=10 tid=0xde723400 nid=0x1f50 in 
Object.wait() [0xdd10f000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '6'" daemon prio=10 tid=0xde721c00 nid=0x1f4f in 
Object.wait() [0xdd160000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '5'" daemon prio=10 tid=0xde720400 nid=0x1f4e in 
Object.wait() [0xdd1b1000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '4'" daemon prio=10 tid=0xde71ec00 nid=0x1f4d in 
Object.wait() [0xdd439000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '3'" daemon prio=10 tid=0xde71e000 nid=0x1f4c in 
Object.wait() [0xdd202000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '2'" daemon prio=10 tid=0xde71c800 nid=0x1f4b in 
Object.wait() [0xdd253000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '1'" daemon prio=10 tid=0xde71b000 nid=0x1f48 in 
Object.wait() [0xdd346000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete thread '0'" daemon prio=10 tid=0xde719800 nid=0x1f47 in 
Object.wait() [0xdd397000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb3068> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100)
	- locked <0xe3fb3068> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:89)

"Document delete stuffer thread" daemon prio=10 tid=0xde718400 
nid=0x1f44 in Object.wait() [0xdd48a000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:919)
	- locked <0xf2706a80> (a java.lang.Integer)
	at 
org.apache.manifoldcf.crawler.system.DocumentDeleteStufferThread.run(DocumentDeleteStufferThread.java:112)

"Document cleanup thread '9'" daemon prio=10 tid=0xde717000 nid=0x1f43 
in Object.wait() [0xdd4db000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '8'" daemon prio=10 tid=0xde715c00 nid=0x1f42 
in Object.wait() [0xdd52c000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '7'" daemon prio=10 tid=0xde715400 nid=0x1f41 
in Object.wait() [0xdde22000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '6'" daemon prio=10 tid=0xde713c00 nid=0x1f40 
in Object.wait() [0xdd57d000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '5'" daemon prio=10 tid=0xde712400 nid=0x1f3f 
in Object.wait() [0xdd5ce000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '4'" daemon prio=10 tid=0xde711000 nid=0x1f3e 
in Object.wait() [0xdd61f000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '3'" daemon prio=10 tid=0xde70f800 nid=0x1f3d 
in Object.wait() [0xdd670000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '2'" daemon prio=10 tid=0xde70e000 nid=0x1f3c 
in Object.wait() [0xdd6c1000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '1'" daemon prio=10 tid=0xde70c800 nid=0x1f3b 
in Object.wait() [0xdd712000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup thread '0'" daemon prio=10 tid=0xde70b000 nid=0x1f3a 
in Object.wait() [0xdd763000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fb7778> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fb7778> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:99)

"Document cleanup stuffer thread" daemon prio=10 tid=0xddc0a400 
nid=0x1f39 in Object.wait() [0xdd7b4000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:919)
	- locked <0xf2640178> (a java.lang.Integer)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupStufferThread.run(DocumentCleanupStufferThread.java:114)

"Expiration thread '9'" daemon prio=10 tid=0xddc09400 nid=0x1f38 in 
Object.wait() [0xdd805000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '8'" daemon prio=10 tid=0xddc07c00 nid=0x1f37 in 
Object.wait() [0xdd856000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '7'" daemon prio=10 tid=0xddc06400 nid=0x1f36 in 
Object.wait() [0xdd8a7000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '6'" daemon prio=10 tid=0xddc05000 nid=0x1f35 in 
Object.wait() [0xdd8f8000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '5'" daemon prio=10 tid=0xddc03c00 nid=0x1f34 in 
Object.wait() [0xdd949000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '4'" daemon prio=10 tid=0xddc02000 nid=0x1f33 in 
Object.wait() [0xdd99a000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '3'" daemon prio=10 tid=0x0a496400 nid=0x1f32 in 
Object.wait() [0xdd9eb000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '2'" daemon prio=10 tid=0x0a494c00 nid=0x1f31 in 
Object.wait() [0xdda3c000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '1'" daemon prio=10 tid=0x0a493800 nid=0x1f30 in 
Object.wait() [0xdda8d000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Expiration thread '0'" daemon prio=10 tid=0x0a492400 nid=0x1f2f in 
Object.wait() [0xddade000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xe3fc7060> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:100)
	- locked <0xe3fc7060> (a java.util.ArrayList)
	at 
org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:89)

"Worker thread '24'" daemon prio=10 tid=0x0a56e000 nid=0x1f2e in 
Object.wait() [0xddb2f000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '23'" daemon prio=10 tid=0x0a56c400 nid=0x1f2d in 
Object.wait() [0xddb80000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '22'" daemon prio=10 tid=0x0a56ac00 nid=0x1f2c in 
Object.wait() [0xddd80000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '21'" daemon prio=10 tid=0x0a569400 nid=0x1f2b in 
Object.wait() [0xdddd1000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '20'" daemon prio=10 tid=0xde709400 nid=0x1f2a in 
Object.wait() [0xddbd1000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '19'" daemon prio=10 tid=0x0a567400 nid=0x1f28 in 
Object.wait() [0xdde73000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '18'" daemon prio=10 tid=0x0a5a5c00 nid=0x1f27 in 
Object.wait() [0xde5ba000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '17'" daemon prio=10 tid=0xde704800 nid=0x1f26 in 
Object.wait() [0xddec4000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '16'" daemon prio=10 tid=0x0a5a4800 nid=0x1f25 in 
Object.wait() [0xddf15000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '15'" daemon prio=10 tid=0x0a5a3000 nid=0x1f24 in 
Object.wait() [0xddf66000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '14'" daemon prio=10 tid=0x0a5a1800 nid=0x1f23 in 
Object.wait() [0xddfb7000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '13'" daemon prio=10 tid=0x0a59fc00 nid=0x1f22 in 
Object.wait() [0xde008000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '12'" daemon prio=10 tid=0xde703000 nid=0x1f20 in 
Object.wait() [0xde059000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '11'" daemon prio=10 tid=0x0a25c400 nid=0x1f1f in 
Object.wait() [0xde0aa000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '10'" daemon prio=10 tid=0x0a180800 nid=0x1f1e in 
Object.wait() [0xde0fb000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '9'" daemon prio=10 tid=0xde701400 nid=0x1f1d in 
Object.wait() [0xde14c000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '8'" daemon prio=10 tid=0xdecfec00 nid=0x1f1c in 
Object.wait() [0xde19d000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Thread.join(Thread.java:1186)
	- locked <0xea62bb98> (a 
org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
	at java.lang.Thread.join(Thread.java:1239)
	at 
org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:673)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:709)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.addDocuments(JobManager.java:4089)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.processDocumentReferences(WorkerThread.java:1932)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.addDocumentReference(WorkerThread.java:1487)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector$ProcessActivityLinkHandler.noteDiscoveredLink(WebcrawlerConnector.java:6049)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector$ProcessActivityHTMLHandler.noteAHREF(WebcrawlerConnector.java:6159)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.LinkParseState.noteNonscriptTag(LinkParseState.java:44)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.FormParseState.noteNonscriptTag(FormParseState.java:52)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ScriptParseState.noteTag(ScriptParseState.java:50)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.BasicParseState.dealWithCharacter(BasicParseState.java:225)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.handleHTML(WebcrawlerConnector.java:7047)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.extractLinks(WebcrawlerConnector.java:6011)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1282)
	at 
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)

"Worker thread '7'" daemon prio=10 tid=0xdecfd400 nid=0x1f1b in 
Object.wait() [0xde1ee000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Thread.join(Thread.java:1186)
	- locked <0xea62cb98> (a 
org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
	at java.lang.Thread.join(Thread.java:1239)
	at 
org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:673)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:709)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.jobs.IntrinsicLink.recordReferences(IntrinsicLink.java:263)
	at org.apache.manifoldcf.crawler.jobs.HopCount.doRecord(HopCount.java:409)
	at 
org.apache.manifoldcf.crawler.jobs.HopCount.recordReferences(HopCount.java:380)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.addDocuments(JobManager.java:4119)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.processDocumentReferences(WorkerThread.java:1932)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.addDocumentReference(WorkerThread.java:1487)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector$ProcessActivityLinkHandler.noteDiscoveredLink(WebcrawlerConnector.java:6049)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector$ProcessActivityHTMLHandler.noteAHREF(WebcrawlerConnector.java:6159)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.LinkParseState.noteNonscriptTag(LinkParseState.java:44)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.FormParseState.noteNonscriptTag(FormParseState.java:52)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ScriptParseState.noteTag(ScriptParseState.java:50)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.BasicParseState.dealWithCharacter(BasicParseState.java:225)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.handleHTML(WebcrawlerConnector.java:7047)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.extractLinks(WebcrawlerConnector.java:6011)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1282)
	at 
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)

"Worker thread '6'" daemon prio=10 tid=0xdecfbc00 nid=0x1f1a in 
Object.wait() [0xde23f000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '5'" daemon prio=10 tid=0xdecfa400 nid=0x1f19 in 
Object.wait() [0xde290000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '4'" daemon prio=10 tid=0xdecf6c00 nid=0x1f18 in 
Object.wait() [0xde2e1000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '3'" daemon prio=10 tid=0xdecf5000 nid=0x1f17 in 
Object.wait() [0xde332000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '2'" daemon prio=10 tid=0x0a0f7400 nid=0x1f16 in 
Object.wait() [0xde383000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '1'" daemon prio=10 tid=0x0a0f5800 nid=0x1f15 in 
Object.wait() [0xde3d4000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Worker thread '0'" daemon prio=10 tid=0x0a0f4000 nid=0x1f14 in 
Object.wait() [0xde425000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
	at 
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryHistoryManager.addRow(RepositoryHistoryManager.java:192)
	at 
org.apache.manifoldcf.crawler.repository.RepositoryConnectionManager.recordHistory(RepositoryConnectionManager.java:650)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity.recordActivity(WorkerThread.java:1263)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ThrottledConnection.doneFetch(ThrottledFetcher.java:1844)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.getDocumentVersions(WebcrawlerConnector.java:802)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:321)

"Set priority thread" daemon prio=10 tid=0x0a0f3000 nid=0x1f13 in 
Object.wait() [0xde476000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:919)
	- locked <0xeec0af68> (a java.lang.Integer)
	at 
org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:148)

"Expire stuffer thread" daemon prio=10 tid=0x0a0bbc00 nid=0x1f12 in 
Object.wait() [0xde4c7000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:827)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.getExpiredDocuments(JobManager.java:1698)
	at 
org.apache.manifoldcf.crawler.system.ExpireStufferThread.run(ExpireStufferThread.java:118)

"Stuffer thread" daemon prio=10 tid=0x0a0d6800 nid=0x1f10 in 
Object.wait() [0xde518000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:919)
	- locked <0xf1d0dc50> (a java.lang.Integer)
	at 
org.apache.manifoldcf.crawler.system.StufferThread.run(StufferThread.java:131)

"Job delete thread" daemon prio=10 tid=0x0a0d5000 nid=0x1f0f in 
Object.wait() [0xde569000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.deleteJobsReadyForDelete(JobManager.java:6364)
	at 
org.apache.manifoldcf.crawler.system.JobDeleteThread.run(JobDeleteThread.java:72)

"Job notification thread" daemon prio=10 tid=0x0a0ba800 nid=0x1f0d in 
Object.wait() [0xde60b000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.getJobsReadyForInactivity(JobManager.java:6555)
	at 
org.apache.manifoldcf.crawler.system.JobNotificationThread.run(JobNotificationThread.java:69)

"Finisher thread" daemon prio=10 tid=0x0a0b9800 nid=0x1f0c in 
Object.wait() [0xde65c000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.finishJobs(JobManager.java:6469)
	at 
org.apache.manifoldcf.crawler.system.FinisherThread.run(FinisherThread.java:64)

"Delete startup thread" daemon prio=10 tid=0x0a0b8800 nid=0x1f0b in 
Object.wait() [0xde6ad000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.getJobsReadyForDelete(JobManager.java:5786)
	at 
org.apache.manifoldcf.crawler.system.StartDeleteThread.run(StartDeleteThread.java:83)

"Startup thread" daemon prio=10 tid=0x0a0b7800 nid=0x1f0a in 
Object.wait() [0xde6fe000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.getJobsReadyForStartup(JobManager.java:5856)
	at 
org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:87)

"Job start thread" daemon prio=10 tid=0x0a20d000 nid=0x1f09 in 
Object.wait() [0xde860000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.internalTransactionBegin(Database.java:250)
	at 
org.apache.manifoldcf.core.database.Database.synchronizeTransactions(Database.java:237)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1382)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.crawler.jobs.JobManager.startJobs(JobManager.java:4711)
	at 
org.apache.manifoldcf.crawler.system.JobStartThread.run(JobStartThread.java:68)

"Connection pool reaper" daemon prio=10 tid=0xdeee3800 nid=0x1ee2 
waiting on condition [0xdedf6000]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$ConnectionCloserThread.run(ConnectionPoolManager.java:130)

"Low Memory Detector" daemon prio=10 tid=0xdf104800 nid=0x1ee0 runnable 
[0x00000000]
    java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0xdf102800 nid=0x1edf waiting on 
condition [0x00000000]
    java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0xdf100800 nid=0x1ede waiting on 
condition [0x00000000]
    java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x09f54800 nid=0x1edd waiting on 
condition [0x00000000]
    java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x09f46000 nid=0x1edc in Object.wait() 
[0xdf56d000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
	- locked <0xe3fe4ad8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x09f41800 nid=0x1edb in 
Object.wait() [0xdf5be000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
	- locked <0xe3fb9178> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x09e01c00 nid=0x1ecc in Object.wait() [0xf7148000]
    java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:80)
	- locked <0xe3fe96e8> (a 
org.apache.manifoldcf.core.jdbcpool.ConnectionPool)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
	at 
org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
	at 
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:722)
	at 
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
	at 
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
	at 
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
	at 
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
	at 
org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:221)
	at 
org.apache.manifoldcf.agents.agentmanager.AgentManager.getAllAgents(AgentManager.java:218)
	at 
org.apache.manifoldcf.agents.system.ManifoldCF.startAgents(ManifoldCF.java:147)
	- locked <0xe3fd4028> (a java.util.HashMap)
	at org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:64)
	at 
org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37)
	at org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:101)

"VM Thread" prio=10 tid=0x09f3d800 nid=0x1eda runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x09e09000 nid=0x1ecd runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x09e0a800 nid=0x1ece runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x09e0bc00 nid=0x1ecf runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x09e0d400 nid=0x1ed0 runnable

"GC task thread#4 (ParallelGC)" prio=10 tid=0x09e0ec00 nid=0x1ed1 runnable

"GC task thread#5 (ParallelGC)" prio=10 tid=0x09e10000 nid=0x1ed2 runnable

"GC task thread#6 (ParallelGC)" prio=10 tid=0x09e11800 nid=0x1ed3 runnable

"GC task thread#7 (ParallelGC)" prio=10 tid=0x09e12c00 nid=0x1ed4 runnable

"GC task thread#8 (ParallelGC)" prio=10 tid=0x09e14400 nid=0x1ed5 runnable

"GC task thread#9 (ParallelGC)" prio=10 tid=0x09e15800 nid=0x1ed6 runnable

"GC task thread#10 (ParallelGC)" prio=10 tid=0x09e17000 nid=0x1ed7 runnable

"GC task thread#11 (ParallelGC)" prio=10 tid=0x09e18800 nid=0x1ed8 runnable

"GC task thread#12 (ParallelGC)" prio=10 tid=0x09e19c00 nid=0x1ed9 runnable

"VM Periodic Task Thread" prio=10 tid=0xdf116800 nid=0x1ee1 waiting on 
condition

JNI global references: 1044

Heap
  PSYoungGen      total 86656K, used 62881K [0xeea60000, 0xf3fb0000, 
0xf3fb0000)
   eden space 85952K, 72% used [0xeea60000,0xf2730530,0xf3e50000)
   from space 704K, 86% used [0xf3e50000,0xf3ee8000,0xf3f00000)
   to   space 704K, 0% used [0xf3f00000,0xf3f00000,0xf3fb0000)
  PSOldGen        total 174784K, used 106478K [0xe3fb0000, 0xeea60000, 
0xeea60000)
   object space 174784K, 60% used [0xe3fb0000,0xea7abb78,0xeea60000)
  PSPermGen       total 16384K, used 12215K [0xdffb0000, 0xe0fb0000, 
0xe3fb0000)
   object space 16384K, 74% used [0xdffb0000,0xe0b9dc00,0xe0fb0000)

Erlend

-- 
Erlend GarĂ¥sen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message