lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: HeapedScorerDoc using all my memory
Date Fri, 03 Apr 2009 13:06:55 GMT
Hmmmm, that's odd. how many is "a large number of documents"? And
what is your index size when things to wonky? (approximately)

I can say that other people create very large indexes without this
happening,
but the only thing that says is that this isn't a *known* problem.

Is there any chance you're re-opening the indexwriter without
closing the old one (not that you should, you should reuse the writer,
but I've put a new in the wrong place by mistake a time or two)....

But I confess that I have no idea what happens under the covers when
indexing,
so I'll have to leave any real insights to the folks who know.

Erick

On Fri, Apr 3, 2009 at 8:41 AM, John Byrne <john.byrne@propylon.com> wrote:

> The maximum JVM memory is 2GB. Apparently 1.2GB is being used up by this
> class.
>
> All IndexWriter settings are left as default.
>
> I haven't tried any changes yet, because the problem so far has on ly
> happened in a production environment that I can't mess with. I am planning
> to try reproducing it locally soon, but it takes quite a while before it
> happens.
>
> -John
>
> Erick Erickson wrote:
>
>> How much memory are you allocating for the JVM? And what are your
>> various indexwriter settings (e.g. MaxBufferedDocs, MaxMergeDocs, etc).
>>
>> Have you tried different settings in setRamBufferSizeMB?
>>
>> Best
>> Erick
>>
>> On Fri, Apr 3, 2009 at 7:13 AM, John Byrne <john.byrne@propylon.com>
>> wrote:
>>
>>
>>
>>> Hi, I'm having a problem where the JVM runs out of memory while indexing
>>> a
>>> large number of files. An analysis of the heapdump shows that most of the
>>> memory was taken up with
>>> "org/apache/lucene/util/ScorerDocQueue$HeapedScorerDoc".
>>>
>>> I can't find any leaks in my code so far, and I was wondering, are there
>>> any known problems with this class that could leak memory?
>>>
>>> I'm using version 2.3.1.
>>>
>>> Thanks!
>>>
>>> -John
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>>
>>
>>  ------------------------------------------------------------------------
>>
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com Version: 8.0.238 / Virus Database:
>> 270.11.39/2038 - Release Date: 04/02/09 19:07:00
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message