activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mandar.wanpal" <mandar.wan...@gmail.com>
Subject AMQ halts and crashes after few thousand reqs
Date Fri, 15 Feb 2013 10:44:45 GMT
Hi All,

We are seeing some serious issue with AMQ in few of our load tests. 

We have configured our AMQ with below configs.

Heap size increased 1GB
JMX port opened for AMQ.
jms.prefetchPolicy.all=10
constantPendingMessageLimitStrategy=50.
-XX:Permsize=128m -XX:MaxPermsize=128m

We have AMQ with KahaDB in simple failover settings, so if AMQ1 fails, AMQ2
takes over. Messages are laso not huge in size.

Observations:
1. If Heapsize set to 512 mb, AMQ fails after some 7500 reqs and switches to
AMQ2. AMQ 2 also not able to continue because producer is not able to
initiate proper communication with AMQ2 because of may be the backlog of
messages that AMQ1 didnt accept. 
2. If Heapsize set to 1g, AMQ fails after some 15000 reqs and switches to
AMQ2. AMQ 2 also not able to continue because producer is not able to
initiate proper communication with AMQ2 because of may be the backlog of
messages that AMQ1 didnt accept. 
3. When AMQ fails, we start getting OutOfMemory errors and AMQ starts doing
FullGC continuously. As FullGC halts JVM, AMQ halts and cant do anything.
4. After seeing so many FullGC, we took heapdump and analysed it with
Eclipse. PFA report which states few suspected areas which are causing leak
in AMQ.

AMQ_Leak_Report.pdf
<http://activemq.2283324.n4.nabble.com/file/n4663532/AMQ_Leak_Report.pdf>  

5. Frequency of FullGC increases with time and amount of memory they can
reclaim gets reduced. 

Queries:
What can be the ideal config for AMQ to atleast process upto 1lac reqs
without requiring a restart and without setting heap to some gigantic size.
I have linux with quadcore + 8GB RAM. 



-----
Regards,
Mandar Wanpal
Email: mandar.wanpal@gmail.com
--
View this message in context: http://activemq.2283324.n4.nabble.com/AMQ-halts-and-crashes-after-few-thousand-reqs-tp4663532.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message