activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Márcio Moraes" <mmoraes...@gmail.com>
Subject Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Date Wed, 17 Oct 2007 18:01:46 GMT
Rob,

May I open a issue about it?

Att,

Márcio Moraes

2007/10/15, Márcio Moraes <mmoraes.br@gmail.com>:
>
> Thanks,
>
> You right it does not solve my problem.
>
> I am think to workaround  it with a script to cleanup old data files.
>
> 2007/10/12, Rob Davies < rajdavies@gmail.com>:
> >
> > looks like it might be a bug  - though it doesn't solve your problem
> > - which is another bug btw - will fix
> >
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
> >
> > > I thought found the bug.
> > >
> > > --- DataFileAccessorPool.java ---
> > >
> > >         public void closeDataFileReader(DataFileAccessor reader) {
> > >             openCounter--;
> > >             used = true; // HERE
> > >             if (pool.size() >= maxOpenReadersPerFile || disposed) {
> > >                 reader.dispose();
> > >             } else {
> > >                 pool.add(reader);
> > >             }
> > >         }
> > >
> > > Att,
> > >
> > > Márcio Moraes
> > >
> > > 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
> > >>
> > >> I see ...
> > >>
> > >> The point is it seems not cleanup  until a restart.
> > >>
> > >> I put a log into DataFileAccessorPool.java to see the disposeUnused
> > >> method.
> > >>
> > >> Then i sent a lot of messages, until my data files look like this:
> > >>
> > >> ls -lh data/localhost/journal/
> > >> total 50M
> > >> -rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
> > >> -rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
> > >> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
> > >>
> > >> and
> > >>
> > >> ls -lh data/localhost/kr-store/data/
> > >> total 40M
> > >> -rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
> > >> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> > >> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> > >> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> > >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31 data-topic-subs-
> > >> references-1
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> > >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.admin-Subscriptions
> > >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.negotiation-Subscriptions
> > >> -rw-r--r-- 1 root root  17K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.notification-Subscriptions
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> > >> hash-index-blob_topic___auction.response-Subscriptions
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___ActiveMQ.Agent
> > >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31
> > >> hash-index-topic-data_topic___auction.admin
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.negotiation
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.notification
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> > >> hash-index-topic-data_topic___auction.response
> > >> -rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
> > >> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> > >> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
> > >> references
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57 lock
> > >>
> > >>
> > >> I thought data-3 should be removed, am i right?
> > >>
> > >> But my log says:
> > >>
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> > >> 20506094
> > >> refCount = 89444 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> > >> 21730205
> > >> refCount = 94782 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> > >> DataFileAccessorPool:112  - file:data-4 number = 4 , length =
> > >> 22957070
> > >> refCount = 100132 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> > >> DataFileAccessorPool:112  - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >>
> > >> if pool.isUsed() is true nothing is cleanup.
> > >>
> > >> It seems that the file into the pool never get unused.
> > >>
> > >> Thanks.
> > >>
> > >> Att,
> > >>
> > >> Márcio Moraes
> > >>
> > >>
> > >>
> > >>
> > >> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>
> > >>> that still looks ok - the default data file length is 32mb - I'll
> > >>> expose the property to make that configurable
> > >>>
> > >>>
> > >>> cheers,
> > >>>
> > >>> Rob
> > >>>
> > >>> http://rajdavies.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> > >>>
> > >>>> I build a new version from SNAPSHOT (revision 583595) and run
> > >>>> again my
> > >>>> tests.
> > >>>>
> > >>>> After sent ~15k messages my data dir was 34mb
> > >>>>
> > >>>> I wait 10 minutes but data still with 34mb ...
> > >>>>
> > >>>> then i restart activemq and data dir was now with 20mb
> > >>>>
> > >>>> Is this a ok behavior? How log it takes to clean up? Always do
i
> > >>>> need
> > >>>> restart?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Márcio Moraes
> > >>>>
> > >>>> 2007/10/11, Márcio Moraes <mmoraes.br@gmail.com>:
> > >>>>>
> > >>>>> Ok, I will try and let you know ...
> > >>>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> > >>>>>>
> > >>>>>> ah - in that case - please use a later snapshot - there
was a bug
> > >>>>>> with selectors which is now fixed
> > >>>>>>
> > >>>>>>
> > >>>>>> cheers,
> > >>>>>>
> > >>>>>> Rob
> > >>>>>>
> > >>>>>> http://rajdavies.blogspot.com/
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> > >>>>>>
> > >>>>>>> I think messages have been readed.
> > >>>>>>>
> > >>>>>>> Because i have 2 durable consumers with selector.
> > >>>>>>>
> > >>>>>>> The first consumer/selector accept the messages the
other one
> > >>>>>>> does
> > >>>>>>> not.
> > >>>>>>>
> > >>>>>>> I think message should be considered readed even to
the second
> > >>>>>>> consumer.
> > >>>>>>>
> > >>>>>>> Am i right?
> > >>>>>>>
> > >>>>>>> Att,
> > >>>>>>>
> > >>>>>>> Márcio Moraes
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>>>>>>
> > >>>>>>>> Hi Marcio,
> > >>>>>>>>
> > >>>>>>>> the messages are stored in a journal file log -
the whole log
> > >>>>>>>> will be
> > >>>>>>>> removed when all the messages have been read. This
is done for
> > >>>>>>>> performance
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> cheers,
> > >>>>>>>>
> > >>>>>>>> Rob
> > >>>>>>>>
> > >>>>>>>> http://rajdavies.blogspot.com/
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following
scenario:
> > >>>>>>>>>
> > >>>>>>>>> - Sending N messages to a Durable Topic
> > >>>>>>>>> - Two consumers identified with its clientId
> > >>>>>>>>> - Both has a selector and only one can accept
the test
> > >>>>>>>>> messages
> > >>>>>>>>>
> > >>>>>>>>> All works fine, except the messages are never
removed from the
> > >>>>>>>>> data
> > >>>>>>>>> directory, it grows infinitely.
> > >>>>>>>>>
> > >>>>>>>>> Is it a bug?
> > >>>>>>>>>
> > >>>>>>>>> My work around may be cron a script to clean
that.
> > >>>>>>>>>
> > >>>>>>>>> Att,
> > >>>>>>>>>
> > >>>>>>>>> Márcio Moraes
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message