activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Márcio Moraes" <mmoraes...@gmail.com>
Subject Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Date Thu, 11 Oct 2007 23:05:06 GMT
I thought found the bug.

--- DataFileAccessorPool.java ---

        public void closeDataFileReader(DataFileAccessor reader) {
            openCounter--;
            used = true; // HERE
            if (pool.size() >= maxOpenReadersPerFile || disposed) {
                reader.dispose();
            } else {
                pool.add(reader);
            }
        }

Att,

Márcio Moraes

2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
>
> I see ...
>
> The point is it seems not cleanup  until a restart.
>
> I put a log into DataFileAccessorPool.java to see the disposeUnused
> method.
>
> Then i sent a lot of messages, until my data files look like this:
>
> ls -lh data/localhost/journal/
> total 50M
> -rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
> -rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
>
> and
>
> ls -lh data/localhost/kr-store/data/
> total 40M
> -rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> -rw-r--r-- 1 root root  17M 2007-10-11 18:31 data-topic-subs-references-1
> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> hash-index-blob_topic___auction.admin-Subscriptions
> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> hash-index-blob_topic___auction.negotiation-Subscriptions
> -rw-r--r-- 1 root root  17K 2007-10-11 17:57
> hash-index-blob_topic___auction.notification-Subscriptions
> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> hash-index-blob_topic___auction.response-Subscriptions
> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> hash-index-topic-data_topic___ActiveMQ.Agent
> -rw-r--r-- 1 root root  17M 2007-10-11 18:31
> hash-index-topic-data_topic___auction.admin
> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> hash-index-topic-data_topic___auction.negotiation
> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> hash-index-topic-data_topic___auction.notification
> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> hash-index-topic-data_topic___auction.response
> -rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-references
> -rw-r--r-- 1 root root    0 2007-10-11 17:57 lock
>
>
> I thought data-3 should be removed, am i right?
>
> But my log says:
>
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> DataFileAccessorPool:112  - file:data-4 number = 4 , length = 20506094
> refCount = 89444 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> DataFileAccessorPool:112  - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> DataFileAccessorPool:112  - file:data-4 number = 4 , length = 21730205
> refCount = 94782 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> DataFileAccessorPool:112  - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> DataFileAccessorPool:112  - file:data-4 number = 4 , length = 22957070
> refCount = 100132 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> DataFileAccessorPool:112  - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
>
> if pool.isUsed() is true nothing is cleanup.
>
> It seems that the file into the pool never get unused.
>
> Thanks.
>
> Att,
>
> Márcio Moraes
>
>
>
>
> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> >
> > that still looks ok - the default data file length is 32mb - I'll
> > expose the property to make that configurable
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> >
> > > I build a new version from SNAPSHOT (revision 583595) and run again my
> > > tests.
> > >
> > > After sent ~15k messages my data dir was 34mb
> > >
> > > I wait 10 minutes but data still with 34mb ...
> > >
> > > then i restart activemq and data dir was now with 20mb
> > >
> > > Is this a ok behavior? How log it takes to clean up? Always do i need
> > > restart?
> > >
> > > Thanks,
> > >
> > > Márcio Moraes
> > >
> > > 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
> > >>
> > >> Ok, I will try and let you know ...
> > >>
> > >> Thanks.
> > >>
> > >> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> > >>>
> > >>> ah - in that case - please use a later snapshot - there was a bug
> > >>> with selectors which is now fixed
> > >>>
> > >>>
> > >>> cheers,
> > >>>
> > >>> Rob
> > >>>
> > >>> http://rajdavies.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> > >>>
> > >>>> I think messages have been readed.
> > >>>>
> > >>>> Because i have 2 durable consumers with selector.
> > >>>>
> > >>>> The first consumer/selector accept the messages the other one does
> > >>>> not.
> > >>>>
> > >>>> I think message should be considered readed even to the second
> > >>>> consumer.
> > >>>>
> > >>>> Am i right?
> > >>>>
> > >>>> Att,
> > >>>>
> > >>>> Márcio Moraes
> > >>>>
> > >>>>
> > >>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>>>
> > >>>>> Hi Marcio,
> > >>>>>
> > >>>>> the messages are stored in a journal file log - the whole log
> > >>>>> will be
> > >>>>> removed when all the messages have been read. This is done
for
> > >>>>> performance
> > >>>>>
> > >>>>>
> > >>>>> cheers,
> > >>>>>
> > >>>>> Rob
> > >>>>>
> > >>>>> http://rajdavies.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> > >>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> > >>>>>>
> > >>>>>> - Sending N messages to a Durable Topic
> > >>>>>> - Two consumers identified with its clientId
> > >>>>>> - Both has a selector and only one can accept the test
messages
> > >>>>>>
> > >>>>>> All works fine, except the messages are never removed from
the
> > >>>>>> data
> > >>>>>> directory, it grows infinitely.
> > >>>>>>
> > >>>>>> Is it a bug?
> > >>>>>>
> > >>>>>> My work around may be cron a script to clean that.
> > >>>>>>
> > >>>>>> Att,
> > >>>>>>
> > >>>>>> Márcio Moraes
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message