activemq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARTEMIS-1700) Server stopped responding and killed itself while exiting paging state
Date Tue, 27 Feb 2018 03:05:00 GMT

    [ https://issues.apache.org/jira/browse/ARTEMIS-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377931#comment-16377931
] 

ASF GitHub Bot commented on ARTEMIS-1700:
-----------------------------------------

Github user shoukunhuai commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1899
  
    So it is a mistake to use global thread pool instead of io thread pool for page cursor.
    
    But this does not fix our problem, as you can see 
    ```
    "Thread-274672 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@4e91d63f)"
Id=274703 TIMED_WAITING on java.util.concurrent.CountDownLatch$Sync@5c416651
    	at sun.misc.Unsafe.park(Native Method)
    	-  waiting on java.util.concurrent.CountDownLatch$Sync@5c416651
    	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
    	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
    	at org.apache.activemq.artemis.core.journal.impl.SimpleWaitIOCallback.waitCompletion(SimpleWaitIOCallback.java:73)
    	at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.waitCompletion(OperationContextImpl.java:313)
    	at org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.waitOnOperations(AbstractJournalStorageManager.java:294)
    	at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.storeBookmark(PageCursorProviderImpl.java:539)
    	at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.cleanupComplete(PageCursorProviderImpl.java:431)
    	at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.cleanup(PageCursorProviderImpl.java:383)
    	-  locked org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl@4f8d6d9a
    	at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl$1.run(PageCursorProviderImpl.java:291)
    	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
    	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
    	at org.apache.activemq.artemis.utils.actors.ProcessorBase$ExecutorTask.run(ProcessorBase.java:53)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    	at java.lang.Thread.run(Thread.java:745)
    
    	Number of locked synchronizers = 2
    	- java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@613d010
    	- java.util.concurrent.ThreadPoolExecutor$Worker@4c03ae59
    ```
    When exit paging state, we will store bookmark for each page subscription and wait until
all callbacks done.
    I believe this may happen even running in io thread as long as singleThreadExecutor in
AbstractJournalStrorageManager use thread from global server thread pool.


> Server stopped responding and killed itself while exiting paging state
> ----------------------------------------------------------------------
>
>                 Key: ARTEMIS-1700
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1700
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 2.4.0
>            Reporter: Qihong Xu
>            Priority: Major
>         Attachments: artemis.log
>
>
> We are currently experiencing this error while running stress test on artemis.
>  
> Basic configuration:
> 1 broker ,1 topic, pub-sub mode.
> Journal type = MAPPED. 
> Threadpool max size = 60.
>  
> In order to test the throughput of artemis we use 300 producers and 300 consumers. However
we found that sometimes when artemis exit paging state, it will stop responding and kill itself.
This situatuion happened on some specific servers.
>  
> Details can be found in attached dump file.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message