lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Olenin" <VOle...@cihi.ca>
Subject RE: Intermittent search performance problem
Date Mon, 06 Nov 2006 16:02:10 GMT
Any profiler can add it's own overhead. You might try the "-verbose:gc"
JVM flag (if you haven't tried it yet). The fastest way to check if you
problems are GC related. Check JVM docs (or plainly - 'java -help' for
more flags. There are some '-X' flags for more detailed info, as well as
flags to dump GC output to a separate file). Watch for 'Tenured
generation' collection ("Full GC"), which are 'stop the world' ones.

All in all, I agree with Yonik and am pretty positive that it's a full
GC. I was actually getting similar pauses time on a similar heap size
with a similar application (custom made 'all in memory' search engine,
so I'd guess the number of object references and their complexity should
be of a similar order).

Also, turn off explicit GC (-XX:+DisableExplicitGC) to disable any
'System.gc()' calls. Though log dump should be sufficient, this is just
another way to make sure if it's a GC that gives you the problems (eg,
if you set -Xmx to 1Gb and you current heap size is 700M AND you are
experiencing pauses without hitting the top limit, it MIGHT indicate
that it's not GC related issue, but generally does NOT guarantee it. GC
Log is the safest and fastest way to sort this kind of problems out).

Vlad

-----Original Message-----
From: Ben Dotte [mailto:ben.dotte@gmail.com] 
Sent: Friday, November 03, 2006 4:48 PM
To: java-user@lucene.apache.org
Subject: Re: Intermittent search performance problem

Good suggestion, I tried watching the GCs in YourKit while testing but
unfortunately they don't seem to line up with the searches that take
forever. They also don't last long enough to make up that kind of time.
I have our heap limited to 1GB right now and its using around 768MB of
that.

On 11/3/06, Ben Dotte <ben.dotte@gmail.com> wrote:
> I'm trying to figure out a way to troubleshoot a performance problem 
> we're seeing when searching against a memory-based index. What happens

> is we will run a search against the index and it generally returns in
> 1 second or less. But every once in a while it takes 15-20 seconds for

> the exact same search for no apparent reason.

Are you sure it's not just a big GC?  How big is your heap?

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search
server

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message