lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Dotte" <ben.do...@gmail.com>
Subject Re: Intermittent search performance problem
Date Tue, 07 Nov 2006 19:52:43 GMT
Thanks for the additional GC suggestions, that does sound like a
likely culprit the more I look into it. It takes a while to get the
app to a state where the problem is reproducible so it will take a
while to fully test the theory. I'll continue investigating the GCs
this week and hopefully get to the bottom of this sooner or later.

Thanks again,
Ben

On 11/6/06, Vladimir Olenin <VOlenin@cihi.ca> wrote:
> Any profiler can add it's own overhead. You might try the "-verbose:gc"
> JVM flag (if you haven't tried it yet). The fastest way to check if you
> problems are GC related. Check JVM docs (or plainly - 'java -help' for
> more flags. There are some '-X' flags for more detailed info, as well as
> flags to dump GC output to a separate file). Watch for 'Tenured
> generation' collection ("Full GC"), which are 'stop the world' ones.
>
> All in all, I agree with Yonik and am pretty positive that it's a full
> GC. I was actually getting similar pauses time on a similar heap size
> with a similar application (custom made 'all in memory' search engine,
> so I'd guess the number of object references and their complexity should
> be of a similar order).
>
> Also, turn off explicit GC (-XX:+DisableExplicitGC) to disable any
> 'System.gc()' calls. Though log dump should be sufficient, this is just
> another way to make sure if it's a GC that gives you the problems (eg,
> if you set -Xmx to 1Gb and you current heap size is 700M AND you are
> experiencing pauses without hitting the top limit, it MIGHT indicate
> that it's not GC related issue, but generally does NOT guarantee it. GC
> Log is the safest and fastest way to sort this kind of problems out).
>
> Vlad
>
> -----Original Message-----
> From: Ben Dotte [mailto:ben.dotte@gmail.com]
> Sent: Friday, November 03, 2006 4:48 PM
> To: java-user@lucene.apache.org
> Subject: Re: Intermittent search performance problem
>
> Good suggestion, I tried watching the GCs in YourKit while testing but
> unfortunately they don't seem to line up with the searches that take
> forever. They also don't last long enough to make up that kind of time.
> I have our heap limited to 1GB right now and its using around 768MB of
> that.
>
> On 11/3/06, Ben Dotte <ben.dotte@gmail.com> wrote:
> > I'm trying to figure out a way to troubleshoot a performance problem
> > we're seeing when searching against a memory-based index. What happens
>
> > is we will run a search against the index and it generally returns in
> > 1 second or less. But every once in a while it takes 15-20 seconds for
>
> > the exact same search for no apparent reason.
>
> Are you sure it's not just a big GC?  How big is your heap?
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene search
> server
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message