activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: ActiveMQ 5.5.1 Kaha File Cleanup Issue
Date Wed, 24 Jul 2013 04:13:33 GMT
Maybe someone else has an explanation.. but wonder if the index and
transaction log became out of sync somehow and a restart kicked it in the
pants sufficiently? There has been a lot of hardening around KahadB since
5.5.1.. Can you reproduce it in any reliable way?


On Tue, Jul 23, 2013 at 11:18 AM, Greenbean <urciolo2@gmail.com> wrote:

> We have encountered a problem where kaha files are build up on occasion.
> Restarting AMQ seems to allow cleanup to occur.  There is a log entry
> before
> the restart which explains contained ACKs are preventing cleanup.  Just
> curious how the restart works around this contained ack issue.
>
> Example recent persisted message files on a server:
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul  9 17:12 db-62483.log
> 44M -rw-r--r-- 1 sysuser sysuser  44M Jul  9 17:16 db-62484.log
> 56M -rw-r--r-- 1 sysuser sysuser  56M Jul  9 17:20 db-62485.log
> 36M -rw-r--r-- 1 sysuser sysuser  36M Jul  9 17:34 db-62486.log
>      0 -rw-r--r-- 1 sysuser sysuser    0 Jul 12 02:43 lock
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul 12 02:47 db-62869.log
> 52M -rw-r--r-- 1 sysuser sysuser  52M Jul 12 02:47 db-62870.log
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul 12 02:52 db-62881.log
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul 12 02:52 db-62882.log
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul 12 04:15 db-62961.log
> 55M -rw-r--r-- 1 sysuser sysuser  55M Jul 14 12:18 db-63281.log
> 51M -rw-r--r-- 1 sysuser sysuser  51M Jul 14 12:19 db-63282.log
> 57M -rw-r--r-- 1 sysuser sysuser  57M Jul 14 12:39 db-63283.log
> 57M -rw-r--r-- 1 sysuser sysuser  57M Jul 14 13:00 db-63284.log
> 53M -rw-r--r-- 1 sysuser sysuser  53M Jul 14 13:31 db-63286.log
> 46M -rw-r--r-- 1 sysuser sysuser  46M Jul 14 13:31 db-63287.log
> 54M -rw-r--r-- 1 sysuser sysuser  54M Jul 14 13:37 db-63288.log
> 35M -rw-r--r-- 1 sysuser sysuser  35M Jul 14 14:03 db-63289.log
> 3.2M -rw------- 1 sysuser sysuser 3.2M Jul 14 14:14 db.redo
> 113M -rw------- 1 sysuser sysuser 113M Jul 14 14:14 db.data
> 29M -rw-r--r-- 1 sysuser sysuser  32M Jul 14 14:14 db-63290.log
> ----------------------------------------------------------------
> Reason 62870 is not cleaned up (you can see 62869 is referenced):
> 2013-07-14 14:07:39,335 | TRACE | not removing data file: 62870 as
> contained
> ack(s) refer to referenced file: [62713, 62800, 62801, 62802, 62803, 62804,
> 62791, 62481, 62869, 62809, 62794, 62870, 62797, 62799] |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal
> Checkpoint Worker
> ----------------------------------------------------------------
> I restarted ActiveMQ as we have seen that files get cleaned up on restart.
> Now db-62870.log has been removed:
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul  9 17:12 db-62483.log
> 44M -rw-r--r-- 1 sysuser sysuser  44M Jul  9 17:16 db-62484.log
> 56M -rw-r--r-- 1 sysuser sysuser  56M Jul  9 17:20 db-62485.log
> 36M -rw-r--r-- 1 sysuser sysuser  36M Jul  9 17:34 db-62486.log
> 33M -rw-r--r-- 1 sysuser sysuser  33M Jul 12 02:47 db-62869.log
> 55M -rw-r--r-- 1 sysuser sysuser  55M Jul 14 12:18 db-63281.log
> 57M -rw-r--r-- 1 sysuser sysuser  57M Jul 14 12:39 db-63283.log
> 53M -rw-r--r-- 1 sysuser sysuser  53M Jul 14 13:31 db-63286.log
> 54M -rw-r--r-- 1 sysuser sysuser  54M Jul 14 13:37 db-63288.log
> 35M -rw-r--r-- 1 sysuser sysuser  35M Jul 14 14:03 db-63289.log
>    0 -rw-r--r-- 1 sysuser sysuser    0 Jul 14 14:31 lock
> 3.2M -rw------- 1 sysuser sysuser 3.2M Jul 14 14:36 db.redo
> 113M -rw------- 1 sysuser sysuser 113M Jul 14 14:36 db.data
> 3.0M -rw-r--r-- 1 sysuser sysuser  32M Jul 14 14:36 db-63291.log
> ----------------------------------------------------------------
> This shows up in the activemq.log file:
> 2013-07-14 14:32:14,245 | TRACE | gc candidates: [62870, 62881, 62882,
> 62961, 63282, 63284, 63287, 63290] |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal
> Checkpoint Worker
> 2013-07-14 14:32:14,245 | DEBUG | Cleanup removing the data files: [62870,
> 62881, 62882, 62961, 63282, 63284, 63287, 63290] |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal
> Checkpoint Worker
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-1-Kaha-File-Cleanup-Issue-tp4669615.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message