activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Márcio Moraes" <mmoraes...@gmail.com>
Subject Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Date Mon, 15 Oct 2007 16:12:56 GMT
Thanks,

You right it does not solve my problem.

I am think to workaround  it with a script to cleanup old data files.

2007/10/12, Rob Davies <rajdavies@gmail.com>:
>
> looks like it might be a bug  - though it doesn't solve your problem
> - which is another bug btw - will fix
>
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
>
> > I thought found the bug.
> >
> > --- DataFileAccessorPool.java ---
> >
> >         public void closeDataFileReader(DataFileAccessor reader) {
> >             openCounter--;
> >             used = true; // HERE
> >             if (pool.size() >= maxOpenReadersPerFile || disposed) {
> >                 reader.dispose();
> >             } else {
> >                 pool.add(reader);
> >             }
> >         }
> >
> > Att,
> >
> > Márcio Moraes
> >
> > 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
> >>
> >> I see ...
> >>
> >> The point is it seems not cleanup  until a restart.
> >>
> >> I put a log into DataFileAccessorPool.java to see the disposeUnused
> >> method.
> >>
> >> Then i sent a lot of messages, until my data files look like this:
> >>
> >> ls -lh data/localhost/journal/
> >> total 50M
> >> -rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
> >> -rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
> >> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
> >>
> >> and
> >>
> >> ls -lh data/localhost/kr-store/data/
> >> total 40M
> >> -rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
> >> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> >> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> >> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31 data-topic-subs-
> >> references-1
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> >> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.admin-Subscriptions
> >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.negotiation-Subscriptions
> >> -rw-r--r-- 1 root root  17K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.notification-Subscriptions
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> >> hash-index-blob_topic___auction.response-Subscriptions
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> >> hash-index-topic-data_topic___ActiveMQ.Agent
> >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31
> >> hash-index-topic-data_topic___auction.admin
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> >> hash-index-topic-data_topic___auction.negotiation
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> >> hash-index-topic-data_topic___auction.notification
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> >> hash-index-topic-data_topic___auction.response
> >> -rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
> >> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> >> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
> >> references
> >> -rw-r--r-- 1 root root    0 2007-10-11 17:57 lock
> >>
> >>
> >> I thought data-3 should be removed, am i right?
> >>
> >> But my log says:
> >>
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> >> 20506094
> >> refCount = 89444 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> >> 21730205
> >> refCount = 94782 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> >> 22957070
> >> refCount = 100132 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >>
> >> if pool.isUsed() is true nothing is cleanup.
> >>
> >> It seems that the file into the pool never get unused.
> >>
> >> Thanks.
> >>
> >> Att,
> >>
> >> Márcio Moraes
> >>
> >>
> >>
> >>
> >> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> >>>
> >>> that still looks ok - the default data file length is 32mb - I'll
> >>> expose the property to make that configurable
> >>>
> >>>
> >>> cheers,
> >>>
> >>> Rob
> >>>
> >>> http://rajdavies.blogspot.com/
> >>>
> >>>
> >>>
> >>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> >>>
> >>>> I build a new version from SNAPSHOT (revision 583595) and run
> >>>> again my
> >>>> tests.
> >>>>
> >>>> After sent ~15k messages my data dir was 34mb
> >>>>
> >>>> I wait 10 minutes but data still with 34mb ...
> >>>>
> >>>> then i restart activemq and data dir was now with 20mb
> >>>>
> >>>> Is this a ok behavior? How log it takes to clean up? Always do i
> >>>> need
> >>>> restart?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Márcio Moraes
> >>>>
> >>>> 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
> >>>>>
> >>>>> Ok, I will try and let you know ...
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> >>>>>>
> >>>>>> ah - in that case - please use a later snapshot - there was
a bug
> >>>>>> with selectors which is now fixed
> >>>>>>
> >>>>>>
> >>>>>> cheers,
> >>>>>>
> >>>>>> Rob
> >>>>>>
> >>>>>> http://rajdavies.blogspot.com/
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> >>>>>>
> >>>>>>> I think messages have been readed.
> >>>>>>>
> >>>>>>> Because i have 2 durable consumers with selector.
> >>>>>>>
> >>>>>>> The first consumer/selector accept the messages the other
one
> >>>>>>> does
> >>>>>>> not.
> >>>>>>>
> >>>>>>> I think message should be considered readed even to the
second
> >>>>>>> consumer.
> >>>>>>>
> >>>>>>> Am i right?
> >>>>>>>
> >>>>>>> Att,
> >>>>>>>
> >>>>>>> Márcio Moraes
> >>>>>>>
> >>>>>>>
> >>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> >>>>>>>>
> >>>>>>>> Hi Marcio,
> >>>>>>>>
> >>>>>>>> the messages are stored in a journal file log - the
whole log
> >>>>>>>> will be
> >>>>>>>> removed when all the messages have been read. This is
done for
> >>>>>>>> performance
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> cheers,
> >>>>>>>>
> >>>>>>>> Rob
> >>>>>>>>
> >>>>>>>> http://rajdavies.blogspot.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following
scenario:
> >>>>>>>>>
> >>>>>>>>> - Sending N messages to a Durable Topic
> >>>>>>>>> - Two consumers identified with its clientId
> >>>>>>>>> - Both has a selector and only one can accept the
test
> >>>>>>>>> messages
> >>>>>>>>>
> >>>>>>>>> All works fine, except the messages are never removed
from the
> >>>>>>>>> data
> >>>>>>>>> directory, it grows infinitely.
> >>>>>>>>>
> >>>>>>>>> Is it a bug?
> >>>>>>>>>
> >>>>>>>>> My work around may be cron a script to clean that.
> >>>>>>>>>
> >>>>>>>>> Att,
> >>>>>>>>>
> >>>>>>>>> Márcio Moraes
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message