lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Dunn <james_h_d...@yahoo.com>
Subject RE: Memory usage
Date Wed, 26 May 2004 20:08:01 GMT
Will,

Thanks for your response.  It may be an object leak. 
I will look into that.

I just ran some more tests and this time I create a
20GB index by repeatedly merging my large index into
itself.

When I ran my test query against that index I got an
OutOfMemoryError on the very first query.  I have my
heap set to 512MB.  Should a query against a 20GB
index require that much memory?  I page through the
results 100 at a time, so I should never have more
than 100 Document objects in memory.  

Any help would be appreciated, thanks!

Jim
--- wallen@Cyveillance.com wrote:
> This sounds like a memory leakage situation.  If you
> are using tomcat I
> would suggest you make sure you are on a recent
> version, as it is known to
> have some memory leaks in version 4.  It doesn't
> make sense that repeated
> queries would use more memory that the most
> demanding query unless objects
> are not getting freed from memory.
> 
> -Will
> 
> -----Original Message-----
> From: James Dunn [mailto:james_h_dunn@yahoo.com]
> Sent: Wednesday, May 26, 2004 3:02 PM
> To: lucene-user@jakarta.apache.org
> Subject: Memory usage
> 
> 
> Hello,
> 
> I was wondering if anyone has had problems with
> memory
> usage and MultiSearcher.
> 
> My index is composed of two sub-indexes that I
> search
> with a MultiSearcher.  The total size of the index
> is
> about 3.7GB with the larger sub-index being 3.6GB
> and
> the smaller being 117MB.
> 
> I am using Lucene 1.3 Final with the compound file
> format.
> 
> Also I search across about 50 fields but I don't use
> wildcard or range queries. 
> 
> Doing repeated searches in this way seems to
> eventually chew up about 500MB of memory which seems
> excessive to me.
> 
> Does anyone have any ideas where I could look to
> reduce the memory my queries consume?
> 
> Thanks,
> 
> Jim
> 
> 
> 	
> 		
> __________________________________
> Do you Yahoo!?
> Friends.  Fun.  Try the all-new Yahoo! Messenger.
> http://messenger.yahoo.com/ 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message