activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hidekazu <hi_de...@yahoo.co.jp>
Subject AcativeMQ becomes slow suddenly in 60 minutes after starting a performance test
Date Fri, 30 Jun 2017 11:10:00 GMT
Hi,

We did a performance test for ActiveMQ and came across a problem.
I hope someone help us understand what was wrong with my ActiveMQ.
Let me explain it here.

<problem>
Transaction time became long suddenly from 50 millsec to 60 sec in about 60
minutes 
after starting a performance test.
ActiveMQ kept cpu and memory usage in a normal condition during the
performance test.
However, iowait rose 10% in 60 minutes.


<environment>
Our ActiveMQ version 5.14.3

We emulated 1,000 IoT clients and sent data to ActiveMQ at one minute
interval.
Payload size sent by IoT client was about 2 Kbytes.
KahaDB size is enough big. (100GB)


<KahaDB log>
There was an error and out of memory in kahaDB log.

--------------------------
2017-06-16 04:39:19,615 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 04:39:19,615 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 04:45:08,527 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 04:45:08,527 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 04:57:16,863 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 04:57:16,863 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 05:10:08,083 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 05:10:08,083 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 05:22:49,960 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 05:22:49,960 [eckpoint Worker] WARN  MessageDatabase               
- Failed to load next journal location: null
2017-06-16 08:19:45,059 [eckpoint Worker] ERROR MessageDatabase               
- Checkpoint failed
java.lang.OutOfMemoryError: GC overhead limit exceeded
2017-06-16 08:19:45,059 [eckpoint Worker] ERROR MessageDatabase               
- Checkpoint failed
java.lang.OutOfMemoryError: GC overhead limit exceeded
--------------------------

Thanks.
Hidekazu



--
View this message in context: http://activemq.2283324.n4.nabble.com/AcativeMQ-becomes-slow-suddenly-in-60-minutes-after-starting-a-performance-test-tp4728103.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message