Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E22B7200C31 for ; Wed, 8 Mar 2017 14:04:59 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id DDCFD160B83; Wed, 8 Mar 2017 13:04:59 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AA1EE160B75 for ; Wed, 8 Mar 2017 14:04:56 +0100 (CET) Received: (qmail 78852 invoked by uid 500); 8 Mar 2017 13:04:54 -0000 Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@manifoldcf.apache.org Delivered-To: mailing list dev@manifoldcf.apache.org Received: (qmail 78839 invoked by uid 99); 8 Mar 2017 13:04:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Mar 2017 13:04:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7AE7EC34C6 for ; Wed, 8 Mar 2017 13:04:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.951 X-Spam-Level: * X-Spam-Status: No, score=1.951 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_NUMSUBJECT=0.5, RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id yoKoDOBGt8qT for ; Wed, 8 Mar 2017 13:04:43 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 247455FCDA for ; Wed, 8 Mar 2017 13:04:43 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 50DB2E0041 for ; Wed, 8 Mar 2017 13:04:38 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id ECC6E24168 for ; Wed, 8 Mar 2017 13:04:37 +0000 (UTC) Date: Wed, 8 Mar 2017 13:04:37 +0000 (UTC) From: "Karl Wright (JIRA)" To: dev@manifoldcf.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (CONNECTORS-1395) Unexpected jobqueue status - record id 1488898668325, expecting active status, saw 4 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 08 Mar 2017 13:05:00 -0000 [ https://issues.apache.org/jira/browse/CONNECTORS-1395?page=3Dcom.atla= ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId= =3D15901174#comment-15901174 ]=20 Karl Wright edited comment on CONNECTORS-1395 at 3/8/17 1:04 PM: ----------------------------------------------------------------- Thread dump is consistent with a lock that was taken but hasn't been freed,= and was dropped on the floor. Probably this happened when the "FATAL" log= entry was created; during the Idle Cleanup thread a lock is getting freed = that was never grabbed. {code} Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode): "Set priority thread-EventThread" #30657286 daemon prio=3D10 os_prio=3D2 ti= d=3D0x000000006e79b800 nid=3D0x149c waiting on condition [0x0000000054cee00= 0] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f710cc40> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Set priority thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30657285 daemon pri= o=3D10 os_prio=3D2 tid=3D0x000000006e79d000 nid=3D0xd4c runnable [0x0000000= 05830f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f710fdc0> (a sun.nio.ch.Util$2) - locked <0x00000000f710fdb0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f710fc88> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Agents thread-EventThread" #30657284 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 000000056dbb800 nid=3D0xab4 waiting on condition [0x0000000057dff000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7112c40> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Agents thread-SendThread(127.0.0.1:8349)" #30657283 daemon prio=3D5 os_pri= o=3D0 tid=3D0x000000006b2e3800 nid=3D0x11e8 runnable [0x000000005f96f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f710df30> (a sun.nio.ch.Util$2) - locked <0x00000000f710df20> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f710dea0> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Idle cleanup thread-EventThread" #30657282 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006c1b8000 nid=3D0x928 waiting on condition [0x000000005f24f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7113470> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Idle cleanup thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30657281 daemon pri= o=3D5 os_prio=3D0 tid=3D0x000000006c1b7000 nid=3D0x1040 runnable [0x0000000= 055f8e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f71144d0> (a sun.nio.ch.Util$2) - locked <0x00000000f71144c0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f7113f90> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Finisher thread-EventThread" #30650263 daemon prio=3D5 os_prio=3D0 tid=3D0= x000000005768f800 nid=3D0x163c waiting on condition [0x000000005ef2f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f70e6bb0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Finisher thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30650262 daemon prio=3D= 5 os_prio=3D0 tid=3D0x000000005768e800 nid=3D0x1158 runnable [0x000000005f0= 2e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f70e9d30> (a sun.nio.ch.Util$2) - locked <0x00000000f70e9d20> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f70e9bf8> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job delete thread-EventThread" #30650034 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006b2e4800 nid=3D0x10e8 waiting on condition [0x000000005f82f000= ] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7098c48> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job delete thread-SendThread(127.0.0.1:8349)" #30650033 daemon prio=3D5 os= _prio=3D0 tid=3D0x000000006c1b8800 nid=3D0x8fc runnable [0x00000000566af000= ] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f709ba78> (a sun.nio.ch.Util$2) - locked <0x00000000f709ba68> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f709b9e8> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job reset thread-EventThread" #30650028 daemon prio=3D5 os_prio=3D0 tid=3D= 0x000000006e79c800 nid=3D0x17a8 waiting on condition [0x000000005c89e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f709dcd0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job reset thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30650027 daemon prio= =3D5 os_prio=3D0 tid=3D0x000000006e79b000 nid=3D0xcec runnable [0x000000005= c4df000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f709e9d8> (a sun.nio.ch.Util$2) - locked <0x00000000f709e9c8> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f709e530> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job reset thread-EventThread" #30649986 daemon prio=3D5 os_prio=3D0 tid=3D= 0x0000000056f19800 nid=3D0x1490 waiting on condition [0x000000005f42f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f70a6aa8> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job reset thread-SendThread(127.0.0.1:8349)" #30649985 daemon prio=3D5 os_= prio=3D0 tid=3D0x0000000056f12800 nid=3D0xacc runnable [0x000000005619e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f70aab20> (a sun.nio.ch.Util$2) - locked <0x00000000f70aab10> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f70aaa90> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Agents thread-EventThread" #30640967 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 00000006c1b6800 nid=3D0x1120 waiting on condition [0x000000005fafe000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6fe3c30> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Agents thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30640966 daemon prio=3D5 = os_prio=3D0 tid=3D0x000000006c1b5800 nid=3D0x12e0 runnable [0x000000005d28f= 000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6fe6db0> (a sun.nio.ch.Util$2) - locked <0x00000000f6fe6da0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6fe6c78> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Set priority thread-EventThread" #30595106 daemon prio=3D10 os_prio=3D2 ti= d=3D0x0000000056c28800 nid=3D0x15f8 waiting on condition [0x000000005fc0f00= 0] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f63607e0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Set priority thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30595105 daemon pri= o=3D10 os_prio=3D2 tid=3D0x0000000056c28000 nid=3D0x12f0 runnable [0x000000= 005fd8f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6362598> (a sun.nio.ch.Util$2) - locked <0x00000000f6362588> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6362318> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Idle cleanup thread-EventThread" #30594300 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006b2e1800 nid=3D0x10c8 waiting on condition [0x000000005fedf000= ] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6367fa0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Idle cleanup thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30594299 daemon pri= o=3D5 os_prio=3D0 tid=3D0x000000006b2e2000 nid=3D0x1540 runnable [0x0000000= 05f68f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6368a58> (a sun.nio.ch.Util$2) - locked <0x00000000f6368a48> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6368740> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Expire stuffer thread-EventThread" #30594028 daemon prio=3D6 os_prio=3D0 t= id=3D0x00000000577bb000 nid=3D0x157c waiting on condition [0x000000005eddf0= 00] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6369b98> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Expire stuffer thread-SendThread(127.0.0.1:8349)" #30594027 daemon prio=3D= 6 os_prio=3D0 tid=3D0x00000000550c9000 nid=3D0x118c runnable [0x00000000580= 4f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f63786f0> (a sun.nio.ch.Util$2) - locked <0x00000000f63786e0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6378398> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Document cleanup stuffer thread-EventThread" #30594022 daemon prio=3D5 os_= prio=3D0 tid=3D0x0000000056f13000 nid=3D0x1634 waiting on condition [0x0000= 00005ec0e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f637e0a0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Document cleanup stuffer thread-SendThread(0:0:0:0:0:0:0:1:8349)" #3059402= 1 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056f18800 nid=3D0x1488 runnabl= e [0x000000005646e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f637b120> (a sun.nio.ch.Util$2) - locked <0x00000000f637b110> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f637b090> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Connection pool reaper" #2098506 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 0006e79a000 nid=3D0xf58 waiting on condition [0x000000006cf3f000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$Connec= tionCloserThread.run(ConnectionPoolManager.java:152) "Assessment thread" #138 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa68= 00 nid=3D0xd90 in Object.wait() [0x000000005e2ef000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff522940> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.AssessmentThread.run(Assess= mentThread.java:68) "Idle cleanup thread" #137 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa= 5800 nid=3D0xc94 in Object.wait() [0x000000005e91f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff575968> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.IdleCleanupThread.run(IdleC= leanupThread.java:74) "Seeding thread" #136 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa5000 = nid=3D0xd94 in Object.wait() [0x000000005e6ee000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000feea03e0> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SeedingThread.run(SeedingTh= read.java:104) "Job reset thread" #135 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa400= 0 nid=3D0x810 in Object.wait() [0x000000005dedf000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff3cd530> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobResetThread.run(JobReset= Thread.java:126) "Document delete thread '9'" #123 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa3800 nid=3D0x91c in Object.wait() [0x000000005e5df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '8'" #122 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa2800 nid=3D0xd74 in Object.wait() [0x000000005e42f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '7'" #121 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa2000 nid=3D0xd80 in Object.wait() [0x000000005856f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '6'" #120 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa1000 nid=3D0xe4c in Object.wait() [0x000000005e1cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '5'" #119 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa0800 nid=3D0x988 in Object.wait() [0x000000005dfde000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '4'" #118 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f92000 nid=3D0x4e0 in Object.wait() [0x000000005ddbe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '3'" #117 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f91800 nid=3D0xf9c in Object.wait() [0x000000005d08f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '2'" #116 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f90800 nid=3D0xf88 in Object.wait() [0x000000005dc6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '1'" #115 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f90000 nid=3D0x2dc in Object.wait() [0x000000005da7f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '0'" #114 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f8f000 nid=3D0xc74 in Object.wait() [0x000000005d94e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete stuffer thread" #113 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 000000056f8e800 nid=3D0xc48 in Object.wait() [0x000000005ae8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.deletingJobsPresent(Jobs= .java:3133) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextDeletableDo= cuments(JobManager.java:1862) at org.apache.manifoldcf.crawler.system.DocumentDeleteStufferThread= .run(DocumentDeleteStufferThread.java:108) "Document cleanup thread '9'" #134 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8d800 nid=3D0x8dc in Object.wait() [0x000000005d7ef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '8'" #133 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8c800 nid=3D0xe04 in Object.wait() [0x000000005d6cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '7'" #132 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8c000 nid=3D0x4b4 in Object.wait() [0x000000005d4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '6'" #131 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8b000 nid=3D0xd38 in Object.wait() [0x000000005d39e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '5'" #130 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3d800 nid=3D0xec8 in Object.wait() [0x000000005d18f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '4'" #129 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3d000 nid=3D0xee8 in Object.wait() [0x000000005cf1f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '3'" #128 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3c000 nid=3D0x9d8 in Object.wait() [0x000000005cd8e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '2'" #127 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3b800 nid=3D0x2b8 in Object.wait() [0x000000005b7ce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '1'" #126 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3a800 nid=3D0xf84 in Object.wait() [0x000000005cbef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '0'" #125 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3a000 nid=3D0xf4c in Object.wait() [0x000000005c78f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup stuffer thread" #124 daemon prio=3D5 os_prio=3D0 tid=3D0x= 0000000056f39000 nid=3D0xf98 in Object.wait() [0x000000005ca6e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.cleaningJobsPresent(Jobs= .java:3150) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextCleanableDo= cuments(JobManager.java:1618) at org.apache.manifoldcf.crawler.system.DocumentCleanupStufferThrea= d.run(DocumentCleanupStufferThread.java:108) "Expiration thread '9'" #112 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= f37000 nid=3D0xc0c in Object.wait() [0x000000005b9de000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '8'" #111 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c72000 nid=3D0x134 in Object.wait() [0x000000005c64e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '7'" #110 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c71800 nid=3D0xff8 in Object.wait() [0x000000005c06f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '6'" #109 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c70800 nid=3D0x9f0 in Object.wait() [0x000000005c3ce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '5'" #108 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c70000 nid=3D0x498 in Object.wait() [0x000000005c2cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '4'" #107 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6f000 nid=3D0x568 in Object.wait() [0x000000005c1bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '3'" #106 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6e800 nid=3D0xd84 in Object.wait() [0x000000005bf4f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '2'" #105 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6d800 nid=3D0x244 in Object.wait() [0x000000005be0f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '1'" #104 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6d000 nid=3D0xe38 in Object.wait() [0x000000005bc3f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '0'" #103 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6c000 nid=3D0xff4 in Object.wait() [0x000000005bade000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Worker thread '29'" #102 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056c6b= 800 nid=3D0xf48 in Object.wait() [0x000000005b8df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '28'" #101 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bf0= 000 nid=3D0x9b4 in Object.wait() [0x000000005b6cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '27'" #100 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bef= 000 nid=3D0x5fc in Object.wait() [0x0000000059d8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '26'" #99 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bee8= 00 nid=3D0x2a4 in Object.wait() [0x000000005b4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '25'" #98 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bed8= 00 nid=3D0xa20 in Object.wait() [0x000000005b3be000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '24'" #97 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bed0= 00 nid=3D0xa90 in Object.wait() [0x000000005b28e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '23'" #96 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bec0= 00 nid=3D0xe90 in Object.wait() [0x000000005b12f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '22'" #95 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056beb8= 00 nid=3D0x998 in Object.wait() [0x000000005af8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '21'" #94 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bea8= 00 nid=3D0x80c in Object.wait() [0x000000005ad8e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '20'" #93 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bea0= 00 nid=3D0x2e0 in Object.wait() [0x000000005ac0e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '19'" #92 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056be90= 00 nid=3D0x2ec in Object.wait() [0x000000005a1bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '18'" #91 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bde0= 00 nid=3D0xd54 in Object.wait() [0x000000005ab0f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '17'" #90 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdd0= 00 nid=3D0x5a4 in Object.wait() [0x000000005a95f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '16'" #89 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdc8= 00 nid=3D0xc98 in Object.wait() [0x000000005a79f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '15'" #88 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdb8= 00 nid=3D0xb18 in Object.wait() [0x000000005a68e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '14'" #87 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdb0= 00 nid=3D0x8bc in Object.wait() [0x000000005a4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '13'" #86 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bda0= 00 nid=3D0x8d8 in Object.wait() [0x0000000058b3e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '12'" #85 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd98= 00 nid=3D0x668 in Object.wait() [0x000000005a30e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '11'" #84 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd88= 00 nid=3D0xfcc in Object.wait() [0x000000005a0be000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '10'" #83 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd80= 00 nid=3D0xef0 in Object.wait() [0x0000000059fae000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '9'" #82 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd700= 0 nid=3D0x3f0 in Object.wait() [0x0000000059eae000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '8'" #81 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3f80= 0 nid=3D0x290 in Object.wait() [0x0000000059c6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '7'" #80 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3e80= 0 nid=3D0x39c in Object.wait() [0x0000000057f3e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '6'" #79 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3e00= 0 nid=3D0x924 in Object.wait() [0x0000000059a9f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '5'" #78 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3d00= 0 nid=3D0xd60 in Object.wait() [0x00000000598cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '4'" #77 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3c00= 0 nid=3D0x938 in Object.wait() [0x00000000597cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '3'" #76 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3b80= 0 nid=3D0xe7c in Object.wait() [0x00000000596af000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '2'" #75 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3a80= 0 nid=3D0xf30 in Object.wait() [0x000000005952e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '1'" #74 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3a00= 0 nid=3D0x12c in Object.wait() [0x000000005935f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '0'" #73 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3900= 0 nid=3D0x4f4 in Object.wait() [0x000000005919f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "History cleanup thread" #72 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= b38800 nid=3D0xec4 in Object.wait() [0x00000000588ff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ffe18000> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.HistoryCleanupThread.run(Hi= storyCleanupThread.java:78) "Set priority thread" #71 daemon prio=3D10 os_prio=3D2 tid=3D0x0000000056ab= b000 nid=3D0x6bc in Object.wait() [0x0000000058fce000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff4abfe8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPr= iorityThread.java:148) "Expire stuffer thread" #70 daemon prio=3D6 os_prio=3D0 tid=3D0x0000000056a= ba800 nid=3D0xdf8 in Object.wait() [0x0000000058ece000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.activeJobsPresent(Jobs.j= ava:3171) at org.apache.manifoldcf.crawler.jobs.JobManager.getExpiredDocument= s(JobManager.java:2441) at org.apache.manifoldcf.crawler.system.ExpireStufferThread.run(Exp= ireStufferThread.java:121) "Stuffer thread" #69 daemon prio=3D6 os_prio=3D0 tid=3D0x0000000056ab9800 n= id=3D0x990 in Object.wait() [0x0000000058d6e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.activeJobsPresent(Jobs.j= ava:3171) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextDocuments(J= obManager.java:2698) at org.apache.manifoldcf.crawler.system.StufferThread.run(StufferTh= read.java:186) "Job delete thread" #68 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab900= 0 nid=3D0x270 in Object.wait() [0x0000000058c5f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff3f2de0> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobDeleteThread.run(JobDele= teThread.java:78) "Job notification thread" #67 daemon prio=3D5 os_prio=3D0 tid=3D0x000000005= 6ab8000 nid=3D0x820 in Object.wait() [0x0000000058a3f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff5d45b8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobNotificationThread.run(J= obNotificationThread.java:431) "Finisher thread" #66 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab7800 = nid=3D0xda4 in Object.wait() [0x00000000587ff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff451868> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.FinisherThread.run(Finisher= Thread.java:69) "Delete startup thread" #65 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056a= b6800 nid=3D0x580 in Object.wait() [0x000000005867f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff50fe98> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.StartDeleteThread.run(Start= DeleteThread.java:93) "Startup thread" #64 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab6000 n= id=3D0xda0 in Object.wait() [0x000000005846e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockObject.enterWriteLock= (LockObject.java:80) - locked <0x00000000f627b5f8> (a org.apache.manifoldcf.core.lockman= ager.ZooKeeperLockObject) at org.apache.manifoldcf.core.lockmanager.LockGate.enterWriteLock(L= ockGate.java:177) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1473) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:299) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mModification(DBInterfacePostgreSQL.java:661) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mUpdate(DBInterfacePostgreSQL.java:254) at org.apache.manifoldcf.core.database.BaseTable.performUpdate(Base= Table.java:80) at org.apache.manifoldcf.crawler.jobs.Jobs.noteJobStarted(Jobs.java= :2322) at org.apache.manifoldcf.crawler.jobs.JobManager.noteJobStarted(Job= Manager.java:6508) at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupTh= read.java:207) "Job start thread" #63 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab5000= nid=3D0xc60 in Object.wait() [0x00000000581df000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff639398> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobStartThread.run(JobStart= Thread.java:96) "Connection pool reaper" #17 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= aa0000 nid=3D0xa94 waiting on condition [0x00000000578df000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$Connec= tionCloserThread.run(ConnectionPoolManager.java:152) "Agents thread" #16 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd2800 ni= d=3D0xf3c in Object.wait() [0x0000000057aff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff4ddce8> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.ru= n(AgentsDaemon.java:210) "Idle cleanup thread" #15 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd1= 000 nid=3D0xe0c in Object.wait() [0x00000000579df000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff58b430> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCl= eanupThread.java:96) "Service Thread" #9 daemon prio=3D9 os_prio=3D0 tid=3D0x000000005506c800 ni= d=3D0xd2c runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread2" #8 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000054fe800= 0 nid=3D0xcbc waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" #7 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000054fe080= 0 nid=3D0x90c waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" #6 daemon prio=3D9 os_prio=3D2 tid=3D0x000000005352a00= 0 nid=3D0xdb8 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Attach Listener" #5 daemon prio=3D5 os_prio=3D2 tid=3D0x0000000053529000 n= id=3D0xc20 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #4 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000053526000= nid=3D0xdc4 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" #3 daemon prio=3D8 os_prio=3D1 tid=3D0x00000000534c3800 nid=3D0= xef4 in Object.wait() [0x00000000549bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(Unknown Source) - locked <0x00000000f0056410> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) "Reference Handler" #2 daemon prio=3D10 os_prio=3D2 tid=3D0x00000000534ba00= 0 nid=3D0xd68 in Object.wait() [0x000000005484f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source) - locked <0x00000000f0171988> (a java.lang.ref.Reference$Lock) "main" #1 prio=3D5 os_prio=3D0 tid=3D0x000000000231f000 nid=3D0xcf8 in Obje= ct.wait() [0x000000000253f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff556a88> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon.runAgents(Agent= sDaemon.java:115) at org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:64= ) at org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.exe= cute(BaseAgentsInitializationCommand.java:37) at org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93) "VM Thread" os_prio=3D2 tid=3D0x00000000534b6000 nid=3D0xc44 runnable "GC task thread#0 (ParallelGC)" os_prio=3D0 tid=3D0x000000000218d000 nid=3D= 0x1e8 runnable "GC task thread#1 (ParallelGC)" os_prio=3D0 tid=3D0x000000000218e800 nid=3D= 0xdb0 runnable "GC task thread#2 (ParallelGC)" os_prio=3D0 tid=3D0x0000000002190000 nid=3D= 0x900 runnable "GC task thread#3 (ParallelGC)" os_prio=3D0 tid=3D0x0000000002193800 nid=3D= 0xca0 runnable "VM Periodic Task Thread" os_prio=3D2 tid=3D0x000000005506f800 nid=3D0xe74 = waiting on condition JNI global references: 269 Heap PSYoungGen total 86016K, used 77935K [0x00000000fab00000, 0x000000010= 0000000, 0x0000000100000000) eden space 84992K, 91% used [0x00000000fab00000,0x00000000ff6a5b20,0x0000= 0000ffe00000) from space 1024K, 62% used [0x00000000ffe00000,0x00000000ffea0000,0x00000= 000fff00000) to space 1024K, 0% used [0x00000000fff00000,0x00000000fff00000,0x000000= 0100000000) ParOldGen total 175104K, used 116435K [0x00000000f0000000, 0x0000000= 0fab00000, 0x00000000fab00000) object space 175104K, 66% used [0x00000000f0000000,0x00000000f71b4c90,0x0= 0000000fab00000) Metaspace used 19235K, capacity 19524K, committed 20096K, reserved 1= 067008K class space used 1939K, capacity 2034K, committed 2176K, reserved 1048= 576K {code} was (Author: kwright@metacarta.com): Thread dump is consistent with a lock that hasn't been freed. {code} Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode): "Set priority thread-EventThread" #30657286 daemon prio=3D10 os_prio=3D2 ti= d=3D0x000000006e79b800 nid=3D0x149c waiting on condition [0x0000000054cee00= 0] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f710cc40> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Set priority thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30657285 daemon pri= o=3D10 os_prio=3D2 tid=3D0x000000006e79d000 nid=3D0xd4c runnable [0x0000000= 05830f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f710fdc0> (a sun.nio.ch.Util$2) - locked <0x00000000f710fdb0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f710fc88> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Agents thread-EventThread" #30657284 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 000000056dbb800 nid=3D0xab4 waiting on condition [0x0000000057dff000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7112c40> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Agents thread-SendThread(127.0.0.1:8349)" #30657283 daemon prio=3D5 os_pri= o=3D0 tid=3D0x000000006b2e3800 nid=3D0x11e8 runnable [0x000000005f96f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f710df30> (a sun.nio.ch.Util$2) - locked <0x00000000f710df20> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f710dea0> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Idle cleanup thread-EventThread" #30657282 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006c1b8000 nid=3D0x928 waiting on condition [0x000000005f24f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7113470> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Idle cleanup thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30657281 daemon pri= o=3D5 os_prio=3D0 tid=3D0x000000006c1b7000 nid=3D0x1040 runnable [0x0000000= 055f8e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f71144d0> (a sun.nio.ch.Util$2) - locked <0x00000000f71144c0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f7113f90> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Finisher thread-EventThread" #30650263 daemon prio=3D5 os_prio=3D0 tid=3D0= x000000005768f800 nid=3D0x163c waiting on condition [0x000000005ef2f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f70e6bb0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Finisher thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30650262 daemon prio=3D= 5 os_prio=3D0 tid=3D0x000000005768e800 nid=3D0x1158 runnable [0x000000005f0= 2e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f70e9d30> (a sun.nio.ch.Util$2) - locked <0x00000000f70e9d20> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f70e9bf8> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job delete thread-EventThread" #30650034 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006b2e4800 nid=3D0x10e8 waiting on condition [0x000000005f82f000= ] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f7098c48> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job delete thread-SendThread(127.0.0.1:8349)" #30650033 daemon prio=3D5 os= _prio=3D0 tid=3D0x000000006c1b8800 nid=3D0x8fc runnable [0x00000000566af000= ] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f709ba78> (a sun.nio.ch.Util$2) - locked <0x00000000f709ba68> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f709b9e8> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job reset thread-EventThread" #30650028 daemon prio=3D5 os_prio=3D0 tid=3D= 0x000000006e79c800 nid=3D0x17a8 waiting on condition [0x000000005c89e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f709dcd0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job reset thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30650027 daemon prio= =3D5 os_prio=3D0 tid=3D0x000000006e79b000 nid=3D0xcec runnable [0x000000005= c4df000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f709e9d8> (a sun.nio.ch.Util$2) - locked <0x00000000f709e9c8> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f709e530> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Job reset thread-EventThread" #30649986 daemon prio=3D5 os_prio=3D0 tid=3D= 0x0000000056f19800 nid=3D0x1490 waiting on condition [0x000000005f42f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f70a6aa8> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Job reset thread-SendThread(127.0.0.1:8349)" #30649985 daemon prio=3D5 os_= prio=3D0 tid=3D0x0000000056f12800 nid=3D0xacc runnable [0x000000005619e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f70aab20> (a sun.nio.ch.Util$2) - locked <0x00000000f70aab10> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f70aaa90> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Agents thread-EventThread" #30640967 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 00000006c1b6800 nid=3D0x1120 waiting on condition [0x000000005fafe000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6fe3c30> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Agents thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30640966 daemon prio=3D5 = os_prio=3D0 tid=3D0x000000006c1b5800 nid=3D0x12e0 runnable [0x000000005d28f= 000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6fe6db0> (a sun.nio.ch.Util$2) - locked <0x00000000f6fe6da0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6fe6c78> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Set priority thread-EventThread" #30595106 daemon prio=3D10 os_prio=3D2 ti= d=3D0x0000000056c28800 nid=3D0x15f8 waiting on condition [0x000000005fc0f00= 0] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f63607e0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Set priority thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30595105 daemon pri= o=3D10 os_prio=3D2 tid=3D0x0000000056c28000 nid=3D0x12f0 runnable [0x000000= 005fd8f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6362598> (a sun.nio.ch.Util$2) - locked <0x00000000f6362588> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6362318> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Idle cleanup thread-EventThread" #30594300 daemon prio=3D5 os_prio=3D0 tid= =3D0x000000006b2e1800 nid=3D0x10c8 waiting on condition [0x000000005fedf000= ] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6367fa0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Idle cleanup thread-SendThread(0:0:0:0:0:0:0:1:8349)" #30594299 daemon pri= o=3D5 os_prio=3D0 tid=3D0x000000006b2e2000 nid=3D0x1540 runnable [0x0000000= 05f68f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f6368a58> (a sun.nio.ch.Util$2) - locked <0x00000000f6368a48> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6368740> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Expire stuffer thread-EventThread" #30594028 daemon prio=3D6 os_prio=3D0 t= id=3D0x00000000577bb000 nid=3D0x157c waiting on condition [0x000000005eddf0= 00] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6369b98> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Expire stuffer thread-SendThread(127.0.0.1:8349)" #30594027 daemon prio=3D= 6 os_prio=3D0 tid=3D0x00000000550c9000 nid=3D0x118c runnable [0x00000000580= 4f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f63786f0> (a sun.nio.ch.Util$2) - locked <0x00000000f63786e0> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f6378398> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Document cleanup stuffer thread-EventThread" #30594022 daemon prio=3D5 os_= prio=3D0 tid=3D0x0000000056f13000 nid=3D0x1634 waiting on condition [0x0000= 00005ec0e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f637e0a0> (a java.util.concurrent= .locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(Unknown Source) at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) "Document cleanup stuffer thread-SendThread(0:0:0:0:0:0:0:1:8349)" #3059402= 1 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056f18800 nid=3D0x1488 runnabl= e [0x000000005646e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown So= urce) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked <0x00000000f637b120> (a sun.nio.ch.Util$2) - locked <0x00000000f637b110> (a java.util.Collections$Unmodifiable= Set) - locked <0x00000000f637b090> (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnS= ocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 141) "Connection pool reaper" #2098506 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 0006e79a000 nid=3D0xf58 waiting on condition [0x000000006cf3f000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$Connec= tionCloserThread.run(ConnectionPoolManager.java:152) "Assessment thread" #138 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa68= 00 nid=3D0xd90 in Object.wait() [0x000000005e2ef000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff522940> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.AssessmentThread.run(Assess= mentThread.java:68) "Idle cleanup thread" #137 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa= 5800 nid=3D0xc94 in Object.wait() [0x000000005e91f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff575968> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.IdleCleanupThread.run(IdleC= leanupThread.java:74) "Seeding thread" #136 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa5000 = nid=3D0xd94 in Object.wait() [0x000000005e6ee000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000feea03e0> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SeedingThread.run(SeedingTh= read.java:104) "Job reset thread" #135 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056fa400= 0 nid=3D0x810 in Object.wait() [0x000000005dedf000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff3cd530> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobResetThread.run(JobReset= Thread.java:126) "Document delete thread '9'" #123 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa3800 nid=3D0x91c in Object.wait() [0x000000005e5df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '8'" #122 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa2800 nid=3D0xd74 in Object.wait() [0x000000005e42f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '7'" #121 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa2000 nid=3D0xd80 in Object.wait() [0x000000005856f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '6'" #120 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa1000 nid=3D0xe4c in Object.wait() [0x000000005e1cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '5'" #119 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056fa0800 nid=3D0x988 in Object.wait() [0x000000005dfde000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '4'" #118 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f92000 nid=3D0x4e0 in Object.wait() [0x000000005ddbe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '3'" #117 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f91800 nid=3D0xf9c in Object.wait() [0x000000005d08f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '2'" #116 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f90800 nid=3D0xf88 in Object.wait() [0x000000005dc6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '1'" #115 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f90000 nid=3D0x2dc in Object.wait() [0x000000005da7f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete thread '0'" #114 daemon prio=3D5 os_prio=3D0 tid=3D0x00000= 00056f8f000 nid=3D0xc74 in Object.wait() [0x000000005d94e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0216580> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocu= ments(DocumentDeleteQueue.java:100) - locked <0x00000000f0216580> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(Do= cumentDeleteThread.java:95) "Document delete stuffer thread" #113 daemon prio=3D5 os_prio=3D0 tid=3D0x0= 000000056f8e800 nid=3D0xc48 in Object.wait() [0x000000005ae8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.deletingJobsPresent(Jobs= .java:3133) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextDeletableDo= cuments(JobManager.java:1862) at org.apache.manifoldcf.crawler.system.DocumentDeleteStufferThread= .run(DocumentDeleteStufferThread.java:108) "Document cleanup thread '9'" #134 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8d800 nid=3D0x8dc in Object.wait() [0x000000005d7ef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '8'" #133 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8c800 nid=3D0xe04 in Object.wait() [0x000000005d6cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '7'" #132 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8c000 nid=3D0x4b4 in Object.wait() [0x000000005d4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '6'" #131 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f8b000 nid=3D0xd38 in Object.wait() [0x000000005d39e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '5'" #130 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3d800 nid=3D0xec8 in Object.wait() [0x000000005d18f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '4'" #129 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3d000 nid=3D0xee8 in Object.wait() [0x000000005cf1f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '3'" #128 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3c000 nid=3D0x9d8 in Object.wait() [0x000000005cd8e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '2'" #127 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3b800 nid=3D0x2b8 in Object.wait() [0x000000005b7ce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '1'" #126 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3a800 nid=3D0xf84 in Object.wait() [0x000000005cbef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup thread '0'" #125 daemon prio=3D5 os_prio=3D0 tid=3D0x0000= 000056f3a000 nid=3D0xf4c in Object.wait() [0x000000005c78f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f025aae8> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f025aae8> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(D= ocumentCleanupThread.java:106) "Document cleanup stuffer thread" #124 daemon prio=3D5 os_prio=3D0 tid=3D0x= 0000000056f39000 nid=3D0xf98 in Object.wait() [0x000000005ca6e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.cleaningJobsPresent(Jobs= .java:3150) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextCleanableDo= cuments(JobManager.java:1618) at org.apache.manifoldcf.crawler.system.DocumentCleanupStufferThrea= d.run(DocumentCleanupStufferThread.java:108) "Expiration thread '9'" #112 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= f37000 nid=3D0xc0c in Object.wait() [0x000000005b9de000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '8'" #111 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c72000 nid=3D0x134 in Object.wait() [0x000000005c64e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '7'" #110 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c71800 nid=3D0xff8 in Object.wait() [0x000000005c06f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '6'" #109 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c70800 nid=3D0x9f0 in Object.wait() [0x000000005c3ce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '5'" #108 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c70000 nid=3D0x498 in Object.wait() [0x000000005c2cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '4'" #107 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6f000 nid=3D0x568 in Object.wait() [0x000000005c1bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '3'" #106 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6e800 nid=3D0xd84 in Object.wait() [0x000000005bf4f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '2'" #105 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6d800 nid=3D0x244 in Object.wait() [0x000000005be0f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '1'" #104 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6d000 nid=3D0xe38 in Object.wait() [0x000000005bc3f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Expiration thread '0'" #103 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= c6c000 nid=3D0xff4 in Object.wait() [0x000000005bade000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f0275528> (a java.util.ArrayList) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDoc= uments(DocumentCleanupQueue.java:107) - locked <0x00000000f0275528> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThre= ad.java:95) "Worker thread '29'" #102 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056c6b= 800 nid=3D0xf48 in Object.wait() [0x000000005b8df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '28'" #101 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bf0= 000 nid=3D0x9b4 in Object.wait() [0x000000005b6cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '27'" #100 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bef= 000 nid=3D0x5fc in Object.wait() [0x0000000059d8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '26'" #99 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bee8= 00 nid=3D0x2a4 in Object.wait() [0x000000005b4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '25'" #98 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bed8= 00 nid=3D0xa20 in Object.wait() [0x000000005b3be000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '24'" #97 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bed0= 00 nid=3D0xa90 in Object.wait() [0x000000005b28e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '23'" #96 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bec0= 00 nid=3D0xe90 in Object.wait() [0x000000005b12f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '22'" #95 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056beb8= 00 nid=3D0x998 in Object.wait() [0x000000005af8f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '21'" #94 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bea8= 00 nid=3D0x80c in Object.wait() [0x000000005ad8e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '20'" #93 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bea0= 00 nid=3D0x2e0 in Object.wait() [0x000000005ac0e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '19'" #92 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056be90= 00 nid=3D0x2ec in Object.wait() [0x000000005a1bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '18'" #91 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bde0= 00 nid=3D0xd54 in Object.wait() [0x000000005ab0f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '17'" #90 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdd0= 00 nid=3D0x5a4 in Object.wait() [0x000000005a95f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '16'" #89 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdc8= 00 nid=3D0xc98 in Object.wait() [0x000000005a79f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '15'" #88 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdb8= 00 nid=3D0xb18 in Object.wait() [0x000000005a68e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '14'" #87 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bdb0= 00 nid=3D0x8bc in Object.wait() [0x000000005a4fe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '13'" #86 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bda0= 00 nid=3D0x8d8 in Object.wait() [0x0000000058b3e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '12'" #85 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd98= 00 nid=3D0x668 in Object.wait() [0x000000005a30e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '11'" #84 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd88= 00 nid=3D0xfcc in Object.wait() [0x000000005a0be000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '10'" #83 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd80= 00 nid=3D0xef0 in Object.wait() [0x0000000059fae000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '9'" #82 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd700= 0 nid=3D0x3f0 in Object.wait() [0x0000000059eae000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '8'" #81 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3f80= 0 nid=3D0x290 in Object.wait() [0x0000000059c6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '7'" #80 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3e80= 0 nid=3D0x39c in Object.wait() [0x0000000057f3e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '6'" #79 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3e00= 0 nid=3D0x924 in Object.wait() [0x0000000059a9f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '5'" #78 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3d00= 0 nid=3D0xd60 in Object.wait() [0x00000000598cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '4'" #77 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3c00= 0 nid=3D0x938 in Object.wait() [0x00000000597cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '3'" #76 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3b80= 0 nid=3D0xe7c in Object.wait() [0x00000000596af000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '2'" #75 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3a80= 0 nid=3D0xf30 in Object.wait() [0x000000005952e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '1'" #74 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3a00= 0 nid=3D0x12c in Object.wait() [0x000000005935f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "Worker thread '0'" #73 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056b3900= 0 nid=3D0x4f4 in Object.wait() [0x000000005919f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(D= ocumentQueue.java:110) - locked <0x00000000f027d248> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThre= ad.java:121) "History cleanup thread" #72 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= b38800 nid=3D0xec4 in Object.wait() [0x00000000588ff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ffe18000> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.HistoryCleanupThread.run(Hi= storyCleanupThread.java:78) "Set priority thread" #71 daemon prio=3D10 os_prio=3D2 tid=3D0x0000000056ab= b000 nid=3D0x6bc in Object.wait() [0x0000000058fce000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff4abfe8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPr= iorityThread.java:148) "Expire stuffer thread" #70 daemon prio=3D6 os_prio=3D0 tid=3D0x0000000056a= ba800 nid=3D0xdf8 in Object.wait() [0x0000000058ece000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.activeJobsPresent(Jobs.j= ava:3171) at org.apache.manifoldcf.crawler.jobs.JobManager.getExpiredDocument= s(JobManager.java:2441) at org.apache.manifoldcf.crawler.system.ExpireStufferThread.run(Exp= ireStufferThread.java:121) "Stuffer thread" #69 daemon prio=3D6 os_prio=3D0 tid=3D0x0000000056ab9800 n= id=3D0x990 in Object.wait() [0x0000000058d6e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockGate.waitForPermissio= n(LockGate.java:136) - locked <0x00000000f627b588> (a org.apache.manifoldcf.core.lockman= ager.LockGate) at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(Lo= ckGate.java:258) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1522) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:280) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mQuery(DBInterfacePostgreSQL.java:862) at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseT= able.java:236) at org.apache.manifoldcf.crawler.jobs.Jobs.activeJobsPresent(Jobs.j= ava:3171) at org.apache.manifoldcf.crawler.jobs.JobManager.getNextDocuments(J= obManager.java:2698) at org.apache.manifoldcf.crawler.system.StufferThread.run(StufferTh= read.java:186) "Job delete thread" #68 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab900= 0 nid=3D0x270 in Object.wait() [0x0000000058c5f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff3f2de0> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobDeleteThread.run(JobDele= teThread.java:78) "Job notification thread" #67 daemon prio=3D5 os_prio=3D0 tid=3D0x000000005= 6ab8000 nid=3D0x820 in Object.wait() [0x0000000058a3f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff5d45b8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobNotificationThread.run(J= obNotificationThread.java:431) "Finisher thread" #66 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab7800 = nid=3D0xda4 in Object.wait() [0x00000000587ff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff451868> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.FinisherThread.run(Finisher= Thread.java:69) "Delete startup thread" #65 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056a= b6800 nid=3D0x580 in Object.wait() [0x000000005867f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff50fe98> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.StartDeleteThread.run(Start= DeleteThread.java:93) "Startup thread" #64 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab6000 n= id=3D0xda0 in Object.wait() [0x000000005846e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at org.apache.manifoldcf.core.lockmanager.LockObject.enterWriteLock= (LockObject.java:80) - locked <0x00000000f627b5f8> (a org.apache.manifoldcf.core.lockman= ager.ZooKeeperLockObject) at org.apache.manifoldcf.core.lockmanager.LockGate.enterWriteLock(L= ockGate.java:177) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(Bas= eLockManager.java:1473) at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLock= s(BaseLockManager.java:803) at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(= CacheManager.java:299) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjects= AndExecute(CacheManager.java:100) at org.apache.manifoldcf.core.database.Database.executeQuery(Databa= se.java:204) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mModification(DBInterfacePostgreSQL.java:661) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.perfor= mUpdate(DBInterfacePostgreSQL.java:254) at org.apache.manifoldcf.core.database.BaseTable.performUpdate(Base= Table.java:80) at org.apache.manifoldcf.crawler.jobs.Jobs.noteJobStarted(Jobs.java= :2322) at org.apache.manifoldcf.crawler.jobs.JobManager.noteJobStarted(Job= Manager.java:6508) at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupTh= read.java:207) "Job start thread" #63 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056ab5000= nid=3D0xc60 in Object.wait() [0x00000000581df000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff639398> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobStartThread.run(JobStart= Thread.java:96) "Connection pool reaper" #17 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056= aa0000 nid=3D0xa94 waiting on condition [0x00000000578df000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$Connec= tionCloserThread.run(ConnectionPoolManager.java:152) "Agents thread" #16 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd2800 ni= d=3D0xf3c in Object.wait() [0x0000000057aff000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff4ddce8> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.ru= n(AgentsDaemon.java:210) "Idle cleanup thread" #15 daemon prio=3D5 os_prio=3D0 tid=3D0x0000000056bd1= 000 nid=3D0xe0c in Object.wait() [0x00000000579df000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff58b430> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCl= eanupThread.java:96) "Service Thread" #9 daemon prio=3D9 os_prio=3D0 tid=3D0x000000005506c800 ni= d=3D0xd2c runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread2" #8 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000054fe800= 0 nid=3D0xcbc waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" #7 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000054fe080= 0 nid=3D0x90c waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" #6 daemon prio=3D9 os_prio=3D2 tid=3D0x000000005352a00= 0 nid=3D0xdb8 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Attach Listener" #5 daemon prio=3D5 os_prio=3D2 tid=3D0x0000000053529000 n= id=3D0xc20 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #4 daemon prio=3D9 os_prio=3D2 tid=3D0x0000000053526000= nid=3D0xdc4 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" #3 daemon prio=3D8 os_prio=3D1 tid=3D0x00000000534c3800 nid=3D0= xef4 in Object.wait() [0x00000000549bf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(Unknown Source) - locked <0x00000000f0056410> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) "Reference Handler" #2 daemon prio=3D10 os_prio=3D2 tid=3D0x00000000534ba00= 0 nid=3D0xd68 in Object.wait() [0x000000005484f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Unknown Source) at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source) - locked <0x00000000f0171988> (a java.lang.ref.Reference$Lock) "main" #1 prio=3D5 os_prio=3D0 tid=3D0x000000000231f000 nid=3D0xcf8 in Obje= ct.wait() [0x000000000253f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.ja= va:1233) - locked <0x00000000ff556a88> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon.runAgents(Agent= sDaemon.java:115) at org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:64= ) at org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.exe= cute(BaseAgentsInitializationCommand.java:37) at org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93) "VM Thread" os_prio=3D2 tid=3D0x00000000534b6000 nid=3D0xc44 runnable "GC task thread#0 (ParallelGC)" os_prio=3D0 tid=3D0x000000000218d000 nid=3D= 0x1e8 runnable "GC task thread#1 (ParallelGC)" os_prio=3D0 tid=3D0x000000000218e800 nid=3D= 0xdb0 runnable "GC task thread#2 (ParallelGC)" os_prio=3D0 tid=3D0x0000000002190000 nid=3D= 0x900 runnable "GC task thread#3 (ParallelGC)" os_prio=3D0 tid=3D0x0000000002193800 nid=3D= 0xca0 runnable "VM Periodic Task Thread" os_prio=3D2 tid=3D0x000000005506f800 nid=3D0xe74 = waiting on condition JNI global references: 269 Heap PSYoungGen total 86016K, used 77935K [0x00000000fab00000, 0x000000010= 0000000, 0x0000000100000000) eden space 84992K, 91% used [0x00000000fab00000,0x00000000ff6a5b20,0x0000= 0000ffe00000) from space 1024K, 62% used [0x00000000ffe00000,0x00000000ffea0000,0x00000= 000fff00000) to space 1024K, 0% used [0x00000000fff00000,0x00000000fff00000,0x000000= 0100000000) ParOldGen total 175104K, used 116435K [0x00000000f0000000, 0x0000000= 0fab00000, 0x00000000fab00000) object space 175104K, 66% used [0x00000000f0000000,0x00000000f71b4c90,0x0= 0000000fab00000) Metaspace used 19235K, capacity 19524K, committed 20096K, reserved 1= 067008K class space used 1939K, capacity 2034K, committed 2176K, reserved 1048= 576K {code} > Unexpected jobqueue status - record id 1488898668325, expecting active st= atus, saw 4 > -------------------------------------------------------------------------= ----------- > > Key: CONNECTORS-1395 > URL: https://issues.apache.org/jira/browse/CONNECTORS-139= 5 > Project: ManifoldCF > Issue Type: Bug > Components: Framework core > Affects Versions: ManifoldCF 2.6 > Reporter: Karl Wright > Assignee: Karl Wright > Fix For: ManifoldCF 2.7 > > > User saw this in the log, after which the system hung: > {code} > ERROR 2017-03-08 00:25:30,433 (Worker thread '14') - Exception tossed: Un= expected jobqueue status - record id 1488898668325, expecting active status= , saw 4 > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job= queue status - record id 1488898668325, expecting active status, saw 4 > at org.apache.manifoldcf.crawler.jobs.JobQueue.updateComp= letedRecord(JobQueue.java:1019) > at org.apache.manifoldcf.crawler.jobs.JobManager.markDocu= mentCompletedMultiple(JobManager.java:3271) > at org.apache.manifoldcf.crawler.system.WorkerThread.run(= WorkerThread.java:710) > WARN 2017-03-08 00:25:30,449 (Worker thread '23') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:30,449 (Worker thread '24') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:30,464 (Worker thread '9') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:30,464 (Worker thread '0') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:31,900 (Worker thread '11') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:31,900 (Worker thread '29') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:32,867 (Worker thread '10') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:32,867 (Worker thread '2') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:33,335 (Worker thread '8') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:36,642 (Worker thread '20') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:37,422 (Worker thread '21') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:38,280 (Worker thread '22') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:38,280 (Worker thread '3') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:38,280 (Worker thread '5') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:38,826 (Worker thread '28') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:39,045 (Worker thread '13') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:45,425 (Worker thread '4') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:45,425 (Worker thread '15') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:45,425 (Worker thread '17') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:46,392 (Worker thread '25') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:25:46,392 (Worker thread '27') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:11,043 (Worker thread '1') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:35,817 (Worker thread '19') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:35,817 (Worker thread '26') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:36,753 (Worker thread '7') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:39,248 (Worker thread '6') - Service interruption r= eported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:39,248 (Worker thread '18') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > WARN 2017-03-08 00:26:43,129 (Worker thread '16') - Service interruption = reported for job 1488898090224 connection 'web': Job no longer active > FATAL 2017-03-08 00:32:24,819 (Idle cleanup thread) - Error tossed: Can't= release lock we don't hold > java.lang.IllegalStateException: Can't release lock we don't hold=E2=80= =9D > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)