activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From triumvir <peter.na...@alfa.de>
Subject Re: WARN | Store update waiting on 1 replica(s) to catch up to log position 0
Date Fri, 08 Nov 2013 13:45:56 GMT
Hi,

i tried it with the version apache-activemq-5.10-20131106.134045-17-bin
today, no success :-(

WARN | Store update waiting on 1 replica(s) to catch up to log position
12184056. 1 slave nodes attaching.

I use jdk1.6.0_22 (I will try it with the newest version again)

Look at the stacktrace:

2013-11-08 14:39:55
Full thread dump Java HotSpot(TM) Server VM (17.1-b03 mixed mode):

"Attach Listener" daemon prio=3 tid=0x08f73800 nid=0x44 waiting on condition
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"LevelDB store io write" daemon prio=3 tid=0x08212c00 nid=0x43 waiting on
condition [0xd53a1000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xf835bd60> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"ActiveMQ BrokerService[omACS-test-FS] Task-2" daemon prio=3 tid=0x08b2d400
nid=0x36 waiting on condition [0xd53f2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xf34449c8> (a
java.util.concurrent.CountDownLatch$Sync)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
        at
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:253)
        at
org.apache.activemq.leveldb.replicated.MasterLevelDBStore.wal_sync_to(MasterLevelDBStore.scala:429)
        at
org.apache.activemq.leveldb.replicated.MasterLevelDBStore.doStart(MasterLevelDBStore.scala:113)
        at
org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at
org.apache.activemq.leveldb.replicated.ElectingLevelDBStore$$anonfun$start_master$1.apply$mcV$sp(ElectingLevelDBStore.scala:226)
        at
org.fusesource.hawtdispatch.package$$anon$4.run(hawtdispatch.scala:357)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

"hawtdispatch timer" daemon prio=3 tid=0x088ee000 nid=0x34 in Object.wait()
[0xd5494000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xf8318348> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:443)
        at
org.fusesource.hawtdispatch.internal.TimerThread.run(TimerThread.java:188)
        - locked <0xf8318348> (a java.lang.Object)

"hawtdispatch-DEFAULT-8" daemon prio=3 tid=0x08ca5800 nid=0x33 runnable
[0xd54e5000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf8307d50> (a sun.nio.ch.Util$1)
        - locked <0xf8307d60> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf8307d08> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-7" daemon prio=3 tid=0x08ee5400 nid=0x32 runnable
[0xd5536000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf8310188> (a sun.nio.ch.Util$1)
        - locked <0xf8310198> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf8310140> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-6" daemon prio=3 tid=0x08f29800 nid=0x31 runnable
[0xd5587000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf830daf0> (a sun.nio.ch.Util$1)
        - locked <0xf830dae0> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf830d8e0> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-5" daemon prio=3 tid=0x08452400 nid=0x30 runnable
[0xd55d8000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf830e138> (a sun.nio.ch.Util$1)
        - locked <0xf830e128> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf830df28> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-4" daemon prio=3 tid=0x08db3c00 nid=0x2f runnable
[0xd5629000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf830e828> (a sun.nio.ch.Util$1)
        - locked <0xf830e818> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf830e618> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-3" daemon prio=3 tid=0x08f94800 nid=0x2e runnable
[0xd567a000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf831c458> (a sun.nio.ch.Util$1)
        - locked <0xf831c468> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf831c410> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-2" daemon prio=3 tid=0x08bad800 nid=0x2d runnable
[0xd56cb000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf83187e0> (a sun.nio.ch.Util$1)
        - locked <0xf83187f0> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf8318798> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-1" daemon prio=3 tid=0x08db0000 nid=0x2c runnable
[0xd571c000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf8314368> (a sun.nio.ch.Util$1)
        - locked <0xf8314378> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf8314320> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at
org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
        at
org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
        at
org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
        at
org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"main-EventThread" daemon prio=3 tid=0x08f5f800 nid=0x2b waiting on
condition [0xd576d000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xf832c080> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)

"main-SendThread(localhost:2182)" daemon prio=3 tid=0x08f5f000 nid=0x2a
runnable [0xd57be000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xf8318c50> (a sun.nio.ch.Util$1)
        - locked <0xf8318c60> (a java.util.Collections$UnmodifiableSet)
        - locked <0xf8318c08> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

"ZooKeeper state change dispatcher thread" daemon prio=3 tid=0x09036c00
nid=0x29 waiting on condition [0xd5aad000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xf8303ea8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at
org.apache.activemq.leveldb.replicated.groups.ZKClient$StateChangeDispatcher.run(ZKClient.java:347)

"RMI RenewClean-[172.25.0.67:38828]" daemon prio=3 tid=0x08f72000 nid=0x27
in Object.wait() [0xd587e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xf8330070> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0xf8330070> (a java.lang.ref.ReferenceQueue$Lock)
        at
sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
        at java.lang.Thread.run(Thread.java:662)

"RMI Scheduler(0)" daemon prio=3 tid=0x093d0800 nid=0x26 waiting on
condition [0xd58eb000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdae08408> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"GC Daemon" daemon prio=3 tid=0x09027000 nid=0x24 in Object.wait()
[0xd598d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac08048> (a sun.misc.GC$LatencyLock)
        at sun.misc.GC$Daemon.run(GC.java:100)
        - locked <0xdac08048> (a sun.misc.GC$LatencyLock)

"RMI Reaper" prio=3 tid=0x09026000 nid=0x23 in Object.wait() [0xd59de000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac0c028> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0xdac0c028> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
        at java.lang.Thread.run(Thread.java:662)

"RMI TCP Accept-0" daemon prio=3 tid=0x08f20800 nid=0x22 runnable
[0xd5a5c000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0xdac10020> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:662)

"RMI TCP Accept-1099" daemon prio=3 tid=0x08f35400 nid=0x20 runnable
[0xd5afe000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0xdac17fd8> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-21" prio=3 tid=0x08c50000 nid=0x1f waiting on condition
[0xd5bab000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdac00108> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-20" prio=3 tid=0x08b3f000 nid=0x1e waiting on condition
[0xd5bfc000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdac00108> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-19" prio=3 tid=0x08f76000 nid=0x1d waiting on condition
[0xd5c4d000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdac00108> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-18" prio=3 tid=0x08b8f400 nid=0x1c waiting on condition
[0xd5c9e000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdac00108> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-17 Acceptor1 SelectChannelConnector@omACS-test-FS:8161" prio=3
tid=0x08e3fc00 nid=0x1b runnable [0xd5cef000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:145)
        - locked <0xdac10258> (a java.lang.Object)
        at
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-16 Acceptor0 SelectChannelConnector@omACS-test-FS:8161" prio=3
tid=0x09028c00 nid=0x1a waiting for monitor entry [0xd5d40000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:129)
        - waiting to lock <0xdac10258> (a java.lang.Object)
        at
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-15 Selector1" prio=3 tid=0x091eec00 nid=0x19 runnable
[0xd5d91000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xdac082d8> (a sun.nio.ch.Util$1)
        - locked <0xdac082e8> (a java.util.Collections$UnmodifiableSet)
        - locked <0xdac08290> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:662)

"qtp22041176-14 Selector0" prio=3 tid=0x08c53800 nid=0x18 runnable
[0xd5de2000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:170)
        at
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0xdac0c1d0> (a sun.nio.ch.Util$1)
        - locked <0xdac0c1e0> (a java.util.Collections$UnmodifiableSet)
        - locked <0xdac0c188> (a sun.nio.ch.DevPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:662)

"HashSessionScavenger-3" daemon prio=3 tid=0x08dcd000 nid=0x17 in
Object.wait() [0xd5e33000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac14128> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0xdac14128> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"HashSessionScavenger-2" daemon prio=3 tid=0x091eb400 nid=0x16 in
Object.wait() [0xd5e84000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac04110> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0xdac04110> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"HashSessionScavenger-1" daemon prio=3 tid=0x08bc4400 nid=0x15 in
Object.wait() [0xd5efd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac001b8> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0xdac001b8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"HashSessionScavenger-0" daemon prio=3 tid=0x0899cc00 nid=0x14 in
Object.wait() [0xd5f4e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac141e0> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0xdac141e0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"RMI TCP Accept-0" daemon prio=3 tid=0x0833f000 nid=0x12 runnable
[0xd634e000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0xdac142a0> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:662)

"Low Memory Detector" daemon prio=3 tid=0x081a5c00 nid=0x11 runnable
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=3 tid=0x081a3000 nid=0x10 waiting on condition
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=3 tid=0x081a1400 nid=0xf waiting on condition
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=3 tid=0x0819fc00 nid=0xe runnable
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=3 tid=0x0818b400 nid=0xd in Object.wait()
[0xd665e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac00428> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0xdac00428> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=3 tid=0x08189c00 nid=0xc in Object.wait()
[0xfae5a000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0xdac08038> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0xdac08038> (a java.lang.ref.Reference$Lock)

"main" prio=3 tid=0x08070000 nid=0x2 waiting on condition [0xfe30e000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xdac56970> (a
java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
        at
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
        at
org.apache.activemq.leveldb.replicated.ElectingLevelDBStore$$anon$1.start(ElectingLevelDBStore.scala:207)
        at
org.apache.activemq.broker.LockableServiceSupport.preStart(LockableServiceSupport.java:94)
        at
org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:54)
        at
org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:613)
        at
org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:602)
        at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:567)
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:88)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)

"VM Thread" prio=3 tid=0x08187400 nid=0xb runnable

"GC task thread#0 (ParallelGC)" prio=3 tid=0x08078400 nid=0x3 runnable

"GC task thread#1 (ParallelGC)" prio=3 tid=0x08079400 nid=0x4 runnable

"GC task thread#2 (ParallelGC)" prio=3 tid=0x0807a800 nid=0x5 runnable

"GC task thread#3 (ParallelGC)" prio=3 tid=0x0807bc00 nid=0x6 runnable

"GC task thread#4 (ParallelGC)" prio=3 tid=0x0807d000 nid=0x7 runnable

"GC task thread#5 (ParallelGC)" prio=3 tid=0x0807e400 nid=0x8 runnable

"GC task thread#6 (ParallelGC)" prio=3 tid=0x0807fc00 nid=0x9 runnable

"GC task thread#7 (ParallelGC)" prio=3 tid=0x08081000 nid=0xa runnable

"VM Periodic Task Thread" prio=3 tid=0x08383c00 nid=0x13 waiting on
condition

JNI global references: 1229                                                              
                                          



--
View this message in context: http://activemq.2283324.n4.nabble.com/WARN-Store-update-waiting-on-1-replica-s-to-catch-up-to-log-position-0-tp4674001p4674072.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message