Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Received-SPF: pass (athena.apache.org: domain of mmoraes.br@gmail.com
 designates 209.85.198.187 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=GKEef2Z+b/oTi+Bd2bpQknrHHBLIBHKyby/U6VhcOGv5EmS9cViDUx0Amz9krZT2caQUosLlWMNPBSG5jbHsI07zKuNrooPU5sLMPw3JaRaIx8AhLFCpVeHWMl5UvVEIyeQQo0fTEbVvim6T6bM0X8g8vovGwXtatES2dic42x0=
Message-ID: <8f6c1e830710171101k9bcb0d7x9bcff106e6052ab4@mail.gmail.com>
Date: Wed, 17 Oct 2007 15:01:46 -0300
From: "=?ISO-8859-1?Q?M=E1rcio_Moraes?=" <mmoraes.br@gmail.com>
To: dev@activemq.apache.org
Subject: Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old
 messages
In-Reply-To: <8f6c1e830710150912h455309efmd9da0971b74d1d6a@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_30778_1661148.1192644106585"
References: <8f6c1e830710110659u7cbb8637y38d9a9436560cadc@mail.gmail.com>
	 <8f6c1e830710111059g4db8b5f9qe25968df1756ab6c@mail.gmail.com>
	 <038BC6DC-F963-46A7-93C2-D47771C6404C@gmail.com>
	 <8f6c1e830710111142w227eb4f5o66c86c1ff38a7f6b@mail.gmail.com>
	 <8f6c1e830710111256r12767779j5a91e98d83a69c3d@mail.gmail.com>
	 <C7B130A6-21D3-41FC-85FC-8D9272563DA5@gmail.com>
	 <8f6c1e830710111437k37949fefy6fc9c8ade8548b6c@mail.gmail.com>
	 <8f6c1e830710111605l55215b84uba38ce922f970837@mail.gmail.com>
	 <0F32B6BB-B2BD-4661-990C-34194B2EB03C@gmail.com>
	 <8f6c1e830710150912h455309efmd9da0971b74d1d6a@mail.gmail.com>

------=_Part_30778_1661148.1192644106585
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Rob,

May I open a issue about it?

Att,

M=E1rcio Moraes

2007/10/15, M=E1rcio Moraes <mmoraes.br@gmail.com>:
>
> Thanks,
>
> You right it does not solve my problem.
>
> I am think to workaround  it with a script to cleanup old data files.
>
> 2007/10/12, Rob Davies < rajdavies@gmail.com>:
> >
> > looks like it might be a bug  - though it doesn't solve your problem
> > - which is another bug btw - will fix
> >
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 12, 2007, at 12:05 AM, M=E1rcio Moraes wrote:
> >
> > > I thought found the bug.
> > >
> > > --- DataFileAccessorPool.java ---
> > >
> > >         public void closeDataFileReader(DataFileAccessor reader) {
> > >             openCounter--;
> > >             used =3D true; // HERE
> > >             if (pool.size() >=3D maxOpenReadersPerFile || disposed) {
> > >                 reader.dispose();
> > >             } else {
> > >                 pool.add(reader);
> > >             }
> > >         }
> > >
> > > Att,
> > >
> > > M=E1rcio Moraes
> > >
> > > 2007/10/11, M=E1rcio Moraes <mmoraes.br@gmail.com>:
> > >>
> > >> I see ...
> > >>
> > >> The point is it seems not cleanup  until a restart.
> > >>
> > >> I put a log into DataFileAccessorPool.java to see the disposeUnused
> > >> method.
> > >>
> > >> Then i sent a lot of messages, until my data files look like this:
> > >>
> > >> ls -lh data/localhost/journal/
> > >> total 50M
> > >> -rw-r--r-- 1 root root  32M 2007-10-11 18:21 data-3
> > >> -rw-r--r-- 1 root root  32M 2007-10-11 18:31 data-4
> > >> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
> > >>
> > >> and
> > >>
> > >> ls -lh data/localhost/kr-store/data/
> > >> total 40M
> > >> -rw-r--r-- 1 root root  949 2007-10-11 17:57 data-blob-1
> > >> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> > >> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> > >> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> > >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31 data-topic-subs-
> > >> references-1
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> > >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.admin-Subscriptions
> > >> -rw-r--r-- 1 root root  33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.negotiation-Subscriptions
> > >> -rw-r--r-- 1 root root  17K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.notification-Subscriptions
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> > >> hash-index-blob_topic___auction.response-Subscriptions
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___ActiveMQ.Agent
> > >> -rw-r--r-- 1 root root  17M 2007-10-11 18:31
> > >> hash-index-topic-data_topic___auction.admin
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.negotiation
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.notification
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:59
> > >> hash-index-topic-data_topic___auction.response
> > >> -rw-r--r-- 1 root root  510 2007-10-11 17:59 index-blob
> > >> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> > >> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
> > >> references
> > >> -rw-r--r-- 1 root root    0 2007-10-11 17:57 lock
> > >>
> > >>
> > >> I thought data-3 should be removed, am i right?
> > >>
> > >> But my log says:
> > >>
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112  - file:data-4 number =3D 4 , length =3D
> > >> 20506094
> > >> refCount =3D 89444 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112  - file:data-3 number =3D 3 , length =3D
> > >> 33554297
> > >> refCount =3D 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112  - file:data-4 number =3D 4 , length =3D
> > >> 21730205
> > >> refCount =3D 94782 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112  - file:data-3 number =3D 3 , length =3D
> > >> 33554297
> > >> refCount =3D 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> > >> DataFileAccessorPool:112  - file:data-4 number =3D 4 , length =3D
> > >> 22957070
> > >> refCount =3D 100132 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> > >> DataFileAccessorPool:112  - file:data-3 number =3D 3 , length =3D
> > >> 33554297
> > >> refCount =3D 108283 pool.isUsed():true
> > >>
> > >> if pool.isUsed() is true nothing is cleanup.
> > >>
> > >> It seems that the file into the pool never get unused.
> > >>
> > >> Thanks.
> > >>
> > >> Att,
> > >>
> > >> M=E1rcio Moraes
> > >>
> > >>
> > >>
> > >>
> > >> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>
> > >>> that still looks ok - the default data file length is 32mb - I'll
> > >>> expose the property to make that configurable
> > >>>
> > >>>
> > >>> cheers,
> > >>>
> > >>> Rob
> > >>>
> > >>> http://rajdavies.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>> On Oct 11, 2007, at 8:56 PM, M=E1rcio Moraes wrote:
> > >>>
> > >>>> I build a new version from SNAPSHOT (revision 583595) and run
> > >>>> again my
> > >>>> tests.
> > >>>>
> > >>>> After sent ~15k messages my data dir was 34mb
> > >>>>
> > >>>> I wait 10 minutes but data still with 34mb ...
> > >>>>
> > >>>> then i restart activemq and data dir was now with 20mb
> > >>>>
> > >>>> Is this a ok behavior? How log it takes to clean up? Always do i
> > >>>> need
> > >>>> restart?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> M=E1rcio Moraes
> > >>>>
> > >>>> 2007/10/11, M=E1rcio Moraes <mmoraes.br@gmail.com>:
> > >>>>>
> > >>>>> Ok, I will try and let you know ...
> > >>>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> 2007/10/11, Rob Davies <rajdavies@gmail.com>:
> > >>>>>>
> > >>>>>> ah - in that case - please use a later snapshot - there was a bu=
g
> > >>>>>> with selectors which is now fixed
> > >>>>>>
> > >>>>>>
> > >>>>>> cheers,
> > >>>>>>
> > >>>>>> Rob
> > >>>>>>
> > >>>>>> http://rajdavies.blogspot.com/
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Oct 11, 2007, at 6:59 PM, M=E1rcio Moraes wrote:
> > >>>>>>
> > >>>>>>> I think messages have been readed.
> > >>>>>>>
> > >>>>>>> Because i have 2 durable consumers with selector.
> > >>>>>>>
> > >>>>>>> The first consumer/selector accept the messages the other one
> > >>>>>>> does
> > >>>>>>> not.
> > >>>>>>>
> > >>>>>>> I think message should be considered readed even to the second
> > >>>>>>> consumer.
> > >>>>>>>
> > >>>>>>> Am i right?
> > >>>>>>>
> > >>>>>>> Att,
> > >>>>>>>
> > >>>>>>> M=E1rcio Moraes
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>>>>>>
> > >>>>>>>> Hi Marcio,
> > >>>>>>>>
> > >>>>>>>> the messages are stored in a journal file log - the whole log
> > >>>>>>>> will be
> > >>>>>>>> removed when all the messages have been read. This is done for
> > >>>>>>>> performance
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> cheers,
> > >>>>>>>>
> > >>>>>>>> Rob
> > >>>>>>>>
> > >>>>>>>> http://rajdavies.blogspot.com/
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Oct 11, 2007, at 2:59 PM, M=E1rcio Moraes wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> > >>>>>>>>>
> > >>>>>>>>> - Sending N messages to a Durable Topic
> > >>>>>>>>> - Two consumers identified with its clientId
> > >>>>>>>>> - Both has a selector and only one can accept the test
> > >>>>>>>>> messages
> > >>>>>>>>>
> > >>>>>>>>> All works fine, except the messages are never removed from th=
e
> > >>>>>>>>> data
> > >>>>>>>>> directory, it grows infinitely.
> > >>>>>>>>>
> > >>>>>>>>> Is it a bug?
> > >>>>>>>>>
> > >>>>>>>>> My work around may be cron a script to clean that.
> > >>>>>>>>>
> > >>>>>>>>> Att,
> > >>>>>>>>>
> > >>>>>>>>> M=E1rcio Moraes
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>

------=_Part_30778_1661148.1192644106585--