lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alon Muchnick <a...@datonics.com>
Subject Fwd: extensive minor garbage collection when using RAMDirectory on Lucune 3.6.2
Date Wed, 09 Jan 2013 17:34:59 GMT
hi ,
after upgrading to Lucune 3.6.2 i noticed there is an extensive minor
garbage collection operations. once or twice a second , and the amount of
memory being freed is about 600 MB each time for a load of 60 searches per
second :

2013-01-09T18:57:24.350+0200: 174200.121: [GC [PSYoungGen:
630064K->544K(630336K)] 1405948K->776747K(1660992K), 0.0084250 secs]
[Times: user=0.03 sys=0.00, real=0.01 secs]
2013-01-09T18:57:24.785+0200: 174200.555: [GC [PSYoungGen:
629920K->704K(630336K)] 1406123K->777083K(1660992K), 0.0066510 secs]
[Times: user=0.04 sys=0.00, real=0.01 secs]

after some investigation , the caused seemed to be Lucune related , we are
using RAM directory (with about 70000 documents ) on top of tomcat 7.

so i took Lucune source code and integrated it to our application , then
run Jprofiler during some  load for a minute and checked the "application
call tree " for "Garbage collected objects" only .
i found that the 2 Lucune method that produce the most garbage collected
object are (*screen shot attached* ) :

(we have several indexes the below is from one of them )


   - 23.2% - 215 MB - 48,252 alloc.
   org.apache.lucene.search.ExactPhraseScorer.


   - 6.3% - 60,481 kB - 728,414 alloc.
   org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer



looking at the currently live object i see the top 2 classes instance are (
a screen shot is also attached )  :

 org.apache.lucene.search.BooleanScorer$Bucket (instance count )
1311699  (size)
41974368

 org.apache.lucene.search.ScoreDoc (instance count )45846  (size)1100304

we have a system with an earlier version  (tomcat 5.5 and Lucune 3.0.3 our
all other code / jars are the same) which handles the same load and has the
same indexes and the minor gc there occurs once every 1-2 second and the
amount of memory being freed is about 60 MB.

does this behavior is normal ?
does a search should really generate so much short lived objects ( about
5MB for a search ) ?
should we try and use some other RAM bases Index ? ( i know
InstantiatedIndex is deprecated )

thanks in advance .

Alon

Mime
View raw message