lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <e...@ehatchersolutions.com>
Subject Re: Memory usage
Date Wed, 26 May 2004 20:51:24 GMT
How big are your actual Documents?  Are you caching Hits?  It stores, 
internally, up to 200 documents.

	Erik


On May 26, 2004, at 4:08 PM, James Dunn wrote:

> Will,
>
> Thanks for your response.  It may be an object leak.
> I will look into that.
>
> I just ran some more tests and this time I create a
> 20GB index by repeatedly merging my large index into
> itself.
>
> When I ran my test query against that index I got an
> OutOfMemoryError on the very first query.  I have my
> heap set to 512MB.  Should a query against a 20GB
> index require that much memory?  I page through the
> results 100 at a time, so I should never have more
> than 100 Document objects in memory.
>
> Any help would be appreciated, thanks!
>
> Jim
> --- wallen@Cyveillance.com wrote:
>> This sounds like a memory leakage situation.  If you
>> are using tomcat I
>> would suggest you make sure you are on a recent
>> version, as it is known to
>> have some memory leaks in version 4.  It doesn't
>> make sense that repeated
>> queries would use more memory that the most
>> demanding query unless objects
>> are not getting freed from memory.
>>
>> -Will
>>
>> -----Original Message-----
>> From: James Dunn [mailto:james_h_dunn@yahoo.com]
>> Sent: Wednesday, May 26, 2004 3:02 PM
>> To: lucene-user@jakarta.apache.org
>> Subject: Memory usage
>>
>>
>> Hello,
>>
>> I was wondering if anyone has had problems with
>> memory
>> usage and MultiSearcher.
>>
>> My index is composed of two sub-indexes that I
>> search
>> with a MultiSearcher.  The total size of the index
>> is
>> about 3.7GB with the larger sub-index being 3.6GB
>> and
>> the smaller being 117MB.
>>
>> I am using Lucene 1.3 Final with the compound file
>> format.
>>
>> Also I search across about 50 fields but I don't use
>> wildcard or range queries.
>>
>> Doing repeated searches in this way seems to
>> eventually chew up about 500MB of memory which seems
>> excessive to me.
>>
>> Does anyone have any ideas where I could look to
>> reduce the memory my queries consume?
>>
>> Thanks,
>>
>> Jim
>>
>>
>> 	
>> 		
>> __________________________________
>> Do you Yahoo!?
>> Friends.  Fun.  Try the all-new Yahoo! Messenger.
>> http://messenger.yahoo.com/
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> lucene-user-help@jakarta.apache.org
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> lucene-user-help@jakarta.apache.org
>>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message