lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter.synced field accumulates data
Date Thu, 18 Mar 2010 09:26:32 GMT
Thanks!

Mike

On Wed, Mar 17, 2010 at 3:16 PM, Gregor Kaczor <gkaczor@gmx.de> wrote:
> followup in
>
> https://issues.apache.org/jira/browse/LUCENE-2328
>
>
> -------- Original-Nachricht --------
>> Datum: Wed, 17 Mar 2010 14:30:25 -0500
>> Von: Michael McCandless <lucene@mikemccandless.com>
>> An: java-dev@lucene.apache.org
>> Betreff: Re: IndexWriter.synced field accumulates data
>
>> You're right!
>>
>> Really we should delete from sync'd when we delete the files.  We need
>> to tie into IndexFileDeleter for that, maybe moving this set into
>> there.
>>
>> Though in practice the amount of actual RAM used should rarely be an
>> issue?  But we should fix it...
>>
>> Can you open an issue?
>>
>> Mike
>>
>> On Wed, Mar 17, 2010 at 1:15 PM, Gregor Kaczor <gkaczor@gmx.de> wrote:
>> > I am running into a strange OutOfMemoryError. My small test application
>> does index and delete some few files. This is repeated for 60k times.
>>  Optimization is run from every 2k times a file is indexed. Index size is 50KB.
>> I did analyze the HeapDumpFile and realized that IndexWriter.synced field
>> occupied more than half of the heap. That field is a private HashSet
>> without a getter. Its task is to hold files which have been synced already.
>> >
>> > There are two calls to addAll and one call to add on synced but no
>> remove or clear throughout the lifecycle of the IndexWriter instance.
>> >
>> > According to the Eclipse Memory Analyzer synced contains 32618 entries
>> which look like file names "_e065_1.del" or "_e067.cfs"
>> >
>> > The index directory contains 10 files only.
>> >
>> > I guess synced is holding obsolete data
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-dev-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message