activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Date Fri, 12 Oct 2007 05:58:48 GMT
looks like it might be a bug  - though it doesn't solve your problem  
- which is another bug btw - will fix



cheers,

Rob

http://rajdavies.blogspot.com/



On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:

> I thought found the bug.
>
> --- DataFileAccessorPool.java ---
>
>         public void closeDataFileReader(DataFileAccessor reader) {
>             openCounter--;
>             used = true; // HERE
>             if (pool.size() >= maxOpenReadersPerFile || disposed) {
>                 reader.dispose();
>             } else {
>                 pool.add(reader);
>             }
>         }
>
> Att,
>
> Márcio Moraes
>
> 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
>>
>> I see ...
>>
>> The point is it seems not cleanup  until a restart.
>>
>> I put a log into DataFileAccessorPool.java to see the disposeUnused
>> method.
>>
>> Then i sent a lot of messages, until my data files look like this:
>>
>> ls -lh data/localhost/journal/
>> total 50M
>> -rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
>> -rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
>> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
>>
>> and
>>
>> ls -lh data/localhost/kr-store/data/
>> total 40M
>> -rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
>> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
>> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
>> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
>> -rw-r--r-- 1 root root  17M 2007-10-11 18:31 data-topic-subs- 
>> references-1
>> -rw-r--r-- 1 root root    0 2007-10-11 17:57
>> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
>> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
>> hash-index-blob_topic___auction.admin-Subscriptions
>> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
>> hash-index-blob_topic___auction.negotiation-Subscriptions
>> -rw-r--r-- 1 root root  17K 2007-10-11 17:57
>> hash-index-blob_topic___auction.notification-Subscriptions
>> -rw-r--r-- 1 root root    0 2007-10-11 17:59
>> hash-index-blob_topic___auction.response-Subscriptions
>> -rw-r--r-- 1 root root    0 2007-10-11 17:57
>> hash-index-topic-data_topic___ActiveMQ.Agent
>> -rw-r--r-- 1 root root  17M 2007-10-11 18:31
>> hash-index-topic-data_topic___auction.admin
>> -rw-r--r-- 1 root root    0 2007-10-11 17:57
>> hash-index-topic-data_topic___auction.negotiation
>> -rw-r--r-- 1 root root    0 2007-10-11 17:57
>> hash-index-topic-data_topic___auction.notification
>> -rw-r--r-- 1 root root    0 2007-10-11 17:59
>> hash-index-topic-data_topic___auction.response
>> -rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
>> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
>> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs- 
>> references
>> -rw-r--r-- 1 root root    0 2007-10-11 17:57 lock
>>
>>
>> I thought data-3 should be removed, am i right?
>>
>> But my log says:
>>
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>> DataFileAccessorPool:112  - file:data-4 number = 4 , length =  
>> 20506094
>> refCount = 89444 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>> DataFileAccessorPool:112  - file:data-3 number = 3 , length =  
>> 33554297
>> refCount = 108283 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>> DataFileAccessorPool:112  - file:data-4 number = 4 , length =  
>> 21730205
>> refCount = 94782 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>> DataFileAccessorPool:112  - file:data-3 number = 3 , length =  
>> 33554297
>> refCount = 108283 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
>> DataFileAccessorPool:112  - file:data-4 number = 4 , length =  
>> 22957070
>> refCount = 100132 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
>> DataFileAccessorPool:112  - file:data-3 number = 3 , length =  
>> 33554297
>> refCount = 108283 pool.isUsed():true
>>
>> if pool.isUsed() is true nothing is cleanup.
>>
>> It seems that the file into the pool never get unused.
>>
>> Thanks.
>>
>> Att,
>>
>> Márcio Moraes
>>
>>
>>
>>
>> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
>>>
>>> that still looks ok - the default data file length is 32mb - I'll
>>> expose the property to make that configurable
>>>
>>>
>>> cheers,
>>>
>>> Rob
>>>
>>> http://rajdavies.blogspot.com/
>>>
>>>
>>>
>>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
>>>
>>>> I build a new version from SNAPSHOT (revision 583595) and run  
>>>> again my
>>>> tests.
>>>>
>>>> After sent ~15k messages my data dir was 34mb
>>>>
>>>> I wait 10 minutes but data still with 34mb ...
>>>>
>>>> then i restart activemq and data dir was now with 20mb
>>>>
>>>> Is this a ok behavior? How log it takes to clean up? Always do i  
>>>> need
>>>> restart?
>>>>
>>>> Thanks,
>>>>
>>>> Márcio Moraes
>>>>
>>>> 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
>>>>>
>>>>> Ok, I will try and let you know ...
>>>>>
>>>>> Thanks.
>>>>>
>>>>> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
>>>>>>
>>>>>> ah - in that case - please use a later snapshot - there was a bug
>>>>>> with selectors which is now fixed
>>>>>>
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>> http://rajdavies.blogspot.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
>>>>>>
>>>>>>> I think messages have been readed.
>>>>>>>
>>>>>>> Because i have 2 durable consumers with selector.
>>>>>>>
>>>>>>> The first consumer/selector accept the messages the other one
 
>>>>>>> does
>>>>>>> not.
>>>>>>>
>>>>>>> I think message should be considered readed even to the second
>>>>>>> consumer.
>>>>>>>
>>>>>>> Am i right?
>>>>>>>
>>>>>>> Att,
>>>>>>>
>>>>>>> Márcio Moraes
>>>>>>>
>>>>>>>
>>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
>>>>>>>>
>>>>>>>> Hi Marcio,
>>>>>>>>
>>>>>>>> the messages are stored in a journal file log - the whole
log
>>>>>>>> will be
>>>>>>>> removed when all the messages have been read. This is done
for
>>>>>>>> performance
>>>>>>>>
>>>>>>>>
>>>>>>>> cheers,
>>>>>>>>
>>>>>>>> Rob
>>>>>>>>
>>>>>>>> http://rajdavies.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>>>>>>>>>
>>>>>>>>> - Sending N messages to a Durable Topic
>>>>>>>>> - Two consumers identified with its clientId
>>>>>>>>> - Both has a selector and only one can accept the test
 
>>>>>>>>> messages
>>>>>>>>>
>>>>>>>>> All works fine, except the messages are never removed
from the
>>>>>>>>> data
>>>>>>>>> directory, it grows infinitely.
>>>>>>>>>
>>>>>>>>> Is it a bug?
>>>>>>>>>
>>>>>>>>> My work around may be cron a script to clean that.
>>>>>>>>>
>>>>>>>>> Att,
>>>>>>>>>
>>>>>>>>> Márcio Moraes
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>


Mime
View raw message