geronimo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: large-scale app.. random spiking on even static files!
Date Sun, 10 Aug 2008 17:05:31 GMT

On Aug 9, 2008, at 12:09 AM, Pete Clark wrote:

> Finally, here are our startup opts as it pertains to GC if it helps:
>
> -XX:+HeapDumpOnOutOfMemoryError -XX:PermSize=256m -XX:MaxPermSize=256m
> -Xmx5120m -Xms5120m -Xss128k -XX:ParallelGCThreads=20
> -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8
> -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31
> -XX:+AggressiveOpts -Xloggc:/var/tmp/gc.log

Have you attempted to correlate long response times with Garbage  
Collection activity? That would be my initial suspicion. I confess  
that I'm not up-to-date on the latest GC parameters. So, without a bit  
of reading on my part, I'm not entirely sure of the effect of your  
tuning parameters.

Does your application actually require a 5 gig heap?

What's your CPU utilization look like during periods of slow response  
time?

IIUC correctly, the below results are for retrieval of a static text  
file. However, there is additional activity on the server (retrieval  
of Shockwave Flash files and a URL proxy). Is that correct? Is it  
feasible to isolate these functions?

--kevan

>
>
> On Fri, Aug 8, 2008 at 11:59 PM, pc3 <peter.r.clark@gmail.com> wrote:
>>
>> Oh yeah, and we're using tomcat as the servlet engine.. thanks!
>>
>>
>> pc3 wrote:
>>>
>>> hey all -
>>>
>>> so i've spent the past two weeks trying to figure out why my large  
>>> scale
>>> application (200m page views per month) is having some serious, yet
>>> random, spikes in response times.
>>>
>>> we've got an app running under gero 1.1.1, using hibernate, mysql,  
>>> etc.
>>> i've got the response times down in general by adding memcaching  
>>> and other
>>> optimizations but we still see the spikes.. e.g. most of our actions
>>> return in 0.1 sec, but occasionally (and often enough!) .. we'll  
>>> see a
>>> random spike up to 30s from different machines in our cluster.
>>>
>>> so for kicks tonight i tried running a test against all of our app  
>>> servers
>>> not for an action but for a plain .txt file ... and guess what...  
>>> spiky
>>> responses just from the txt file!  a 4.9kb text file!  look:
>>>
>>> $profileUrl /blah/Languages.txt
>>> Host 192.168.1.100
>>> Time taken for tests:   8.555825 seconds <<<<<<<<<<<<
>>> Host 192.168.1.101
>>> Time taken for tests:   0.2064 seconds
>>> Host 192.168.1.108
>>> Time taken for tests:   0.1436 seconds
>>> Host 192.168.1.104
>>> Time taken for tests:   0.1667 seconds
>>> Host 192.168.1.112
>>> Time taken for tests:   0.1444 seconds
>>> Host 192.168.1.117
>>> Time taken for tests:   0.4575 seconds
>>> Host 192.168.1.118
>>> Time taken for tests:   0.2015 seconds
>>> Host 192.168.1.119
>>> Time taken for tests:   0.2003 seconds
>>> Host 192.168.1.120
>>> Time taken for tests:   0.1713 seconds
>>> Host 192.168.1.121
>>> Time taken for tests:   7.22861 seconds <<<<<<<<<<<<
>>> Host 192.168.1.122
>>> Time taken for tests:   0.1615 seconds
>>>
>>> Here are some things to go on:
>>> 1) we still host a few swfs from these servers that are hit  
>>> frequently.
>>> could these be causing some kind of blocking?
>>> 2) we have a method in our application that acts as a proxy,  
>>> taking a url
>>> as a parameter, fetching the url then dumping it to the response
>>>
>>> I'm open to any ideas or suggestions as to what could be causing the
>>> spikiness of response times here for a txt file.. one that never  
>>> even hits
>>> our java code.
>>>
>>> Thanks very much all...
>>>
>>>
>>
>> --
>> View this message in context: http://www.nabble.com/large-scale-app..-random-spiking-on-even-static-files%21-tp18902003s134p18902031.html
>> Sent from the Apache Geronimo - Users mailing list archive at  
>> Nabble.com.
>>
>>


Mime
View raw message