lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Bamford <chris.bamf...@talktalk.net>
Subject Re: RE: Memory question
Date Tue, 15 May 2012 19:38:49 GMT
Thanks Uwe. 

What I'd like to understand is the implications of this on a server which opens a large number
of indexes over a long period. Will this non-heap memory continue to grow? Will gc be effective
at spotting it and releasing it via references in the heap?

 I had an instance yesterday where a server swapped itself to a standstill and had to be restarted.
The load average was through the roof and I am trying to understand why. One of my recent
changes is updating from 2.3 to 3.6, so naturally I am keen to know the impact of the mmap
stuff which is now standard under the covers. 

My server caches indexsearchers and then closes them based on how full the heap is getting.
My worry is that if the bulk of the memory is being allocated outside the Jvm, how can I make
sensible decisions?

Thanks for any pointers / info.

Chris



-----Original Message-----
From: uwe@thetaphi.de
To: java-user@lucene.apache.org
Sent: Tue, 15 May 2012 18:10
Subject: RE: Memory question



It mmaps the files into virtual memory if it runs on a 64 bit JVM. Because
of that you see the mmapped CFS files. This is outside Java Heap and is all
*virtual* no RAM is explicitely occupied except the O/S cache.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Chris Bamford [mailto:chris.bamford@talktalk.net]
> Sent: Tuesday, May 15, 2012 4:47 PM
> To: java-user@lucene.apache.org
> Subject: Memory question
> 
> Hi
> 
> Can anyone tell me what happens to the memory when Lucene opens an index?
> Is it loaded into the JVM's heap or is it mapped into virtual memory
outside of
> it?
> I am running on Linux and if I use pmap on the PID of my JVM, I can see
lots of
> entries for index cfs files.
> 
> Does this mean that indexes are mapped into non-heap memory?  If so, how
> can I monitor the space my process is using if I cache open
IndexSearchers?
> 
> The details are:
> 
> Sun 64-bit JVM on Linux.
> Lucene 3.6 running in 2.3 compatibility mode (as we are in the in the
process of
> a migration to 3.6)
> 
> Thanks,
> 
> - Chris

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message