lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Bamford <chris.bamf...@talktalk.net>
Subject Re: Memory question
Date Tue, 15 May 2012 21:38:00 GMT

 Hi John,

Very interesting, thanks for the detailed explanation.   It certainly sounds like the same
symptoms!
Can I please clarify a couple of things ?

- I have googled CMS / iCMS as I wasn't familiar with those acronyms (apart from 'code management
system' and that didn't sound right!) 
  and am I right in thinking that it is some sort of monitoring code pulled into your server
via a jar? (I'm confused why it would have its' own
  GC cycle...)
- So are you suggesting I play with my own JVM's (Sun/Oracle) parameters to achieve a similar
effect ?

Thanks again,

- Chris


 

 

-----Original Message-----
From: Nader, John P <john.nader@cengage.com>
To: java-user@lucene.apache.org <java-user@lucene.apache.org>
Sent: Tue, 15 May 2012 21:12
Subject: Re: Memory question


We've encountered this issue and came up with a fairly good approach to
address it.

We are on Lucene 3.0.2 with Java 1.6.0_29.  Our indices are about 35GB in
size.  Our JVM runs at 20GB of heap, with about 12GB of steady usage.  Our
server has 32GB total.

What would happen in our case is that the linux would page in more and
more of the memory mapped index files into memory, forcing idle portions
of the JVM heap to be swapped out.  This was not an issue until our CMS GC
kicked in.  This would force swapping in of all JVM memory to collect
unused references.  I/O wait would shoot up and performance would suffer.
Yes, even CMS can kill performance is you are swapping.  The tell-tale
sign was a spike in inbound swap at the start of CMS.

In our case, we addressed the situation using iCMS, which is Incremental
CMS.  This takes the mark phase (and sweep too?) and does it continuously
with a configurable duty cycle.  The result was that swapping was smoothed
out to be a small stead drag on the system instead of a hard spike.  There
was an small loss in performance, but a big gain in stability.

This tuning may be an option for you.  BTW, pmap will give you statistic
on total file size and how much is resident.  The java heap shows up in
pmap as well on linux, so you can determine how much of that is in memory
as well.

John



On 5/15/12 3:38 PM, "Chris Bamford" <chris.bamford@talktalk.net> wrote:

>Thanks Uwe. 
>
>What I'd like to understand is the implications of this on a server which
>opens a large number of indexes over a long period. Will this non-heap
>memory continue to grow? Will gc be effective at spotting it and
>releasing it via references in the heap?
>
> I had an instance yesterday where a server swapped itself to a
>standstill and had to be restarted. The load average was through the roof
>and I am trying to understand why. One of my recent changes is updating
>from 2.3 to 3.6, so naturally I am keen to know the impact of the mmap
>stuff which is now standard under the covers.
>
>My server caches indexsearchers and then closes them based on how full
>the heap is getting. My worry is that if the bulk of the memory is being
>allocated outside the Jvm, how can I make sensible decisions?
>
>Thanks for any pointers / info.
>
>Chris
>
>
>
>-----Original Message-----
>From: uwe@thetaphi.de
>To: java-user@lucene.apache.org
>Sent: Tue, 15 May 2012 18:10
>Subject: RE: Memory question
>
>
>
>It mmaps the files into virtual memory if it runs on a 64 bit JVM. Because
>of that you see the mmapped CFS files. This is outside Java Heap and is
>all
>*virtual* no RAM is explicitely occupied except the O/S cache.
>
>-----
>Uwe Schindler
>H.-H.-Meier-Allee 63, D-28213 Bremen
>http://www.thetaphi.de
>eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Chris Bamford [mailto:chris.bamford@talktalk.net]
>> Sent: Tuesday, May 15, 2012 4:47 PM
>> To: java-user@lucene.apache.org
>> Subject: Memory question
>> 
>> Hi
>> 
>> Can anyone tell me what happens to the memory when Lucene opens an
>>index?
>> Is it loaded into the JVM's heap or is it mapped into virtual memory
>outside of
>> it?
>> I am running on Linux and if I use pmap on the PID of my JVM, I can see
>lots of
>> entries for index cfs files.
>> 
>> Does this mean that indexes are mapped into non-heap memory?  If so, how
>> can I monitor the space my process is using if I cache open
>IndexSearchers?
>> 
>> The details are:
>> 
>> Sun 64-bit JVM on Linux.
>> Lucene 3.6 running in 2.3 compatibility mode (as we are in the in the
>process of
>> a migration to 3.6)
>> 
>> Thanks,
>> 
>> - Chris
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message