lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter.synced field accumulates data
Date Wed, 17 Mar 2010 19:30:25 GMT
You're right!

Really we should delete from sync'd when we delete the files.  We need
to tie into IndexFileDeleter for that, maybe moving this set into
there.

Though in practice the amount of actual RAM used should rarely be an
issue?  But we should fix it...

Can you open an issue?

Mike

On Wed, Mar 17, 2010 at 1:15 PM, Gregor Kaczor <gkaczor@gmx.de> wrote:
> I am running into a strange OutOfMemoryError. My small test application does index and
delete some few files. This is repeated for 60k times.  Optimization is run from every 2k
times a file is indexed. Index size is 50KB. I did analyze the HeapDumpFile and realized that
IndexWriter.synced field occupied more than half of the heap. That field is a private HashSet
without a getter. Its task is to hold files which have been synced already.
>
> There are two calls to addAll and one call to add on synced but no remove or clear throughout
the lifecycle of the IndexWriter instance.
>
> According to the Eclipse Memory Analyzer synced contains 32618 entries which look like
file names "_e065_1.del" or "_e067.cfs"
>
> The index directory contains 10 files only.
>
> I guess synced is holding obsolete data
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message