activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hariharan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-3424) AMQ data dir not cleaning up even after messages getting consumed.
Date Tue, 02 Aug 2011 12:41:27 GMT

    [ https://issues.apache.org/jira/browse/AMQ-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13076170#comment-13076170
] 

Hariharan commented on AMQ-3424:
--------------------------------

I tried using the logger and this is all I see getting logged in kahadb.log.

{code:xml}
 2011-08-02 00:45:24,601 [eckpoint Worker] TRACE MessageDatabase                - Last update:
1:144374, full gc candidates set: [1]
 2011-08-02 00:45:24,601 [eckpoint Worker] TRACE MessageDatabase                - gc candidates
after first tx:1:144374, []
 2011-08-02 00:45:24,601 [eckpoint Worker] TRACE MessageDatabase                - gc candidates:
[]
 2011-08-02 00:45:24,601 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:29,602 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:29,604 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:34,605 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:34,607 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:39,607 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:39,611 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:44,611 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:44,615 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:49,615 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:49,619 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
 2011-08-02 00:45:54,620 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
 2011-08-02 00:45:54,623 [eckpoint Worker] TRACE MessageDatabase                - Last update:
1:155030, full gc candidates set: [1]
 2011-08-02 00:45:54,623 [eckpoint Worker] TRACE MessageDatabase                - gc candidates
after first tx:1:155030, []
 2011-08-02 00:45:54,623 [eckpoint Worker] TRACE MessageDatabase                - gc candidates:
[]
{code}

So I tried recreating this scenario using test cases. Attached are the test cases of producer
and consumer.
Below is the snapshot of the data dir.

{code:xml}
> du -h /activemq_data_kahadb/ ; ls -l /activemq_data_kahadb/
864K    /activemq_data_kahadb/
total 868
-rw-rw-r-- 1 manih manih 9961472 Aug  2 08:26 db-1.log
-rw-rw-r-- 1 manih manih  405504 Aug  2 08:26 db.data
-rw-rw-r-- 1 manih manih  352936 Aug  2 08:26 db.redo
-rw-rw-r-- 1 manih manih       0 Aug  2 08:24 lock

> du -h /activemq_data_kahadb/ ; ls -l /activemq_data_kahadb/
1.1M    /activemq_data_kahadb/
total 1088
-rw-rw-r-- 1 manih manih 9961472 Aug  2 08:28 db-1.log
-rw-rw-r-- 1 manih manih  405504 Aug  2 08:28 db.data
-rw-rw-r-- 1 manih manih  352936 Aug  2 08:28 db.redo
-rw-rw-r-- 1 manih manih       0 Aug  2 08:24 lock

> du -h /activemq_data_kahadb/ ; ls -l /activemq_data_kahadb/
1.3M    /activemq_data_kahadb/
total 1284
-rw-rw-r-- 1 manih manih 9961472 Aug  2 08:29 db-1.log
-rw-rw-r-- 1 manih manih  405504 Aug  2 08:29 db.data
-rw-rw-r-- 1 manih manih  352936 Aug  2 08:29 db.redo
-rw-rw-r-- 1 manih manih       0 Aug  2 08:24 lock
{code}

What I see here is that the file sizes are remaining same while the dir size keeps growing.


Can you please help me understand where the data is going? 
Please let me know if there is any thing wrong in what am trying to do.

> AMQ data dir not cleaning up even after messages getting consumed.
> ------------------------------------------------------------------
>
>                 Key: AMQ-3424
>                 URL: https://issues.apache.org/jira/browse/AMQ-3424
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>            Reporter: Hariharan
>
> Hi,
> We are using ActiveMQ 5.3 (in UNIX) and use both queues and topics (durable and non-durable).
> The messages are published by the publishers in persistent delivery and auto acknowledge
mode.
> I can see that the consumers are actively able to consume the messages. 
> However, even after the messages getting consumed successfully, the data dir size is
not coming down. 
> I tried :
>  1. Using both amqPersistenceAdapter and kahaDB.
>  2. Upgrading the AMQ to 5.5
> But none of them is bringing down its size.
> snapshot:
> {code:xml} 
> *After starting the broker.*
> du -h activemq_data_kahadb ; ls -l /activemq_data_kahadb
> 48K     /activemq_data_kahadb
> total 44
> -rw-rw-r-- 1 manih manih 9961472 Jul 29 05:06 db-1.log
> -rw-rw-r-- 1 manih manih   12288 Jul 29 05:06 db.data
> -rw-rw-r-- 1 manih manih   12304 Jul 29 05:06 db.redo
> -rw-rw-r-- 1 manih manih       0 Jul 29 05:04 lock
> *After messages getting delivered.*
> du -h /activemq_data_kahadb ; ls -l /activemq_data_kahadb
> 1.7M    /activemq_data_kahadb
> total 1652
> -rw-rw-r-- 1 manih manih 9961472 Jul 29 05:24 db-1.log
> -rw-rw-r-- 1 manih manih  364544 Jul 29 05:24 db.data
> -rw-rw-r-- 1 manih manih  332416 Jul 29 05:24 db.redo
> -rw-rw-r-- 1 manih manih       0 Jul 29 05:04 lock
> {code} 
> Here are my config details:
> {code:xml} 
> <amq:persistenceAdapter>
>     <amq:kahaDB directory="activemq_data_kahadb" journalMaxFileLength="10mb"/>
> </amq:persistenceAdapter>
> <amq:systemUsage>
>     <amq:systemUsage>
>         <amq:memoryUsage>
>             <amq:memoryUsage limit="20 mb"/>
>         </amq:memoryUsage>
>         <amq:storeUsage>
>             <amq:storeUsage limit="1 gb"/>
>         </amq:storeUsage>
>         <amq:tempUsage>
>             <amq:tempUsage limit="100 mb"/>
>          </amq:tempUsage>
>     </amq:systemUsage>
> </amq:systemUsage>
> {code} 
> I do understand that the messages are persisted until consumed by the consumers, but
after that shouldn't the file size be coming down?
> Please let me know if there are any config changes that I need to do to get this working
fine.
> Thanks in advance
> Hari

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message