Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <20100317201619.106370@gmx.net>
References: <20100317181547.90050@gmx.net>
	 <9ac0c6aa1003171230k18f74894i1d60f91704e1e62d@mail.gmail.com>
	 <20100317201619.106370@gmx.net>
Date: Thu, 18 Mar 2010 04:26:32 -0500
Message-ID: <9ac0c6aa1003180226n7f9ad939rf55678ff8654072c@mail.gmail.com>
Subject: Re: IndexWriter.synced field accumulates data
From: Michael McCandless <lucene@mikemccandless.com>
To: java-dev@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks!

Mike

On Wed, Mar 17, 2010 at 3:16 PM, Gregor Kaczor <gkaczor@gmx.de> wrote:
> followup in
>
> https://issues.apache.org/jira/browse/LUCENE-2328
>
>
> -------- Original-Nachricht --------
>> Datum: Wed, 17 Mar 2010 14:30:25 -0500
>> Von: Michael McCandless <lucene@mikemccandless.com>
>> An: java-dev@lucene.apache.org
>> Betreff: Re: IndexWriter.synced field accumulates data
>
>> You're right!
>>
>> Really we should delete from sync'd when we delete the files. =A0We need
>> to tie into IndexFileDeleter for that, maybe moving this set into
>> there.
>>
>> Though in practice the amount of actual RAM used should rarely be an
>> issue? =A0But we should fix it...
>>
>> Can you open an issue?
>>
>> Mike
>>
>> On Wed, Mar 17, 2010 at 1:15 PM, Gregor Kaczor <gkaczor@gmx.de> wrote:
>> > I am running into a strange OutOfMemoryError. My small test applicatio=
n
>> does index and delete some few files. This is repeated for 60k times.
>> =A0Optimization is run from every 2k times a file is indexed. Index size=
 is 50KB.
>> I did analyze the HeapDumpFile and realized that IndexWriter.synced fiel=
d
>> occupied more than half of the heap. That field is a private HashSet
>> without a getter. Its task is to hold files which have been synced alrea=
dy.
>> >
>> > There are two calls to addAll and one call to add on synced but no
>> remove or clear throughout the lifecycle of the IndexWriter instance.
>> >
>> > According to the Eclipse Memory Analyzer synced contains 32618 entries
>> which look like file names "_e065_1.del" or "_e067.cfs"
>> >
>> > The index directory contains 10 files only.
>> >
>> > I guess synced is holding obsolete data
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-dev-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org