jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roderick Parks" <roderick.pa...@triometric.net>
Subject RE: BeanShell and JVM Garbage Collection
Date Fri, 22 Jun 2012 15:30:47 GMT
Fair point about being unable to change the default, I'd forgotten that the boolProp might
not be written into the JMX file if the value has been left at the default.

Unfortunately, I think the issue with garbage collection is also due to BeanShell/JMeter interaction
as my script is lightweight.

The gist of my script is to update dates in XML API requests by applying a calculated offset.


StringBuilder sample = new StringBuilder( vars.get("REQUEST") );
replaceDate("FromDate");
replaceDate("ToDate");
...replace various other dates...
vars.put("SAMPLE",sample.toString());
sample = null;
//System.gc();

${REQUEST} is pulled out of the CSV file, ${SAMPLE} is the modified XML used by the sampler.

Putting the script into its own sampler rather than pre-processor, there's no observable difference
in performance or behaviour between using StringBuilder and String, ie. within replaceDate(String
tag):
	sample.replace(dateStart,dateEnd, adjustDate(sample.substring(dateStart,dateEnd)) );
vs
	sample = sample.substring(0,dateStart) + adjustDate(sample.substring(dateStart,dateEnd))
+ sample.substring(dateEnd);

though obviously String adds the previous 'sample' string to the discarded objects for each
date instance that gets replaced whereas StringBuilder does it just the once at the end when
set null (without worrying about how it is actually implemented). As the XML samples can be
quite large and contain many date-related tags, using String would be expected to produce
a lot more garbage than StringBuilder, so some difference in performance or behaviour might
be expected.

Resetting the BeanShell interpreter is expensive in elapsed time, but very cheap compared
to invoking the garbage collector. 

Without reset, average execution time is 9ms, throughput is ~375/sec.
With reset, average execution time is 96ms, throughput is ~99/sec.
With System.gc(), average execution time is 903ms, throughput is 6.6/sec.

The stalling problem is pretty consistent, kicking in at ~6000 iterations on CentOS x86_64,
~40000 iterations on Windows 7 32-bit, so it would be great to be able to reset say every
'n' iterations of a thread to facilitate garbage collection in a timely manner whilst minimising
the performance hit.  My impression is that each BeanShell sampler has its own instance of
the BeanShell Interpreter, because interleaving a copy of the sampler with "Reset bsh.Interpreter"
set every 8th iteration did not change the symptoms: it still stalls when the other 7 samplers
have totalled ~6000 or ~40000 iterations.

If I insert a /* very large comment block */ at the foot of the script, then the problem kicks
in much sooner. Interestingly, the first few stalls are brief: it is as if collecting the
larger garbage is more straightforward, however, the stalls increase in frequency and duration
and it degrades. This must be because the script is passed to the BeanShell Interpreter every
time it is run and ends up in garbage collection too.

For the time being, I'll have to live with the performance hit, though with 500k transactions,
it's an increase from a theoretical 21 minutes to an actual 84 minutes. JMeter is spending
far more time resetting BeanShell Interpreter than replacing the dates or running the samples.


-----Original Message-----
From: sebb [mailto:sebbaz@gmail.com] 
Sent: 21 June 2012 16:55
To: JMeter Users List
Subject: Re: BeanShell and JVM Garbage Collection

On 21 June 2012 16:16, Shmuel Krakower <shmulikk@gmail.com> wrote:
> Yep you got me right, thanks.
>
> So Roderick Parks problem might come from his own code and not from the
> beanshell sampler itself.

Or it could be a combination of the two - i.e. the way he uses
BeanShell might cause the leak; alternate code might avoid it.

> In order to be sure  Roderick should create a heap dump of jmeter when the
> original memory problem occurs and than analyze to see if the leak came

Yes, but given that reset() provides a solution, there's no pressing
need to establish the actual cause.

> from his code / objects or from jmeter's.

We know it does not come from JMeter because resetting the beanshell
server fixes the issue.

> Make sense?
>
> On Thu, Jun 21, 2012 at 6:08 PM, sebb <sebbaz@gmail.com> wrote:
>
>> On 21 June 2012 16:01, Shmuel Krakower <shmulikk@gmail.com> wrote:
>> > Sebb,
>> > Aside of compatibility, does it make sense that the "Reset
>> bsh.Interpreter
>> > before each call" will solve this kind of a problem?
>>
>> Not quite sure what you mean here.
>> The reset() function actually creates a new interpreter, thus allowing
>> GC to tidy up the old interpreter.
>>
>> > And if so, it might be caused by some kind of leak in the beanshell code
>> of
>> > the user and not necessarily due to bug in JMeter, right?
>>
>> Yes, the leak might be in user code or BeanShell itself.
>>
>> And if the leak is in BeanShell itself, maybe it will be fixed one day...
>>
>> > Shmuel.
>> >
>> >
>> > On Thu, Jun 21, 2012 at 5:47 PM, sebb <sebbaz@gmail.com> wrote:
>> >
>> >> On 20 June 2012 18:54, Roderick Parks <roderick.parks@triometric.net>
>> >> wrote:
>> >> > I hit an interesting problem today where my test slowed to a crawl
>> after
>> >> > about just 6000 iterations (out of a total of about 500000 required)
>> >> > irrespective of the number of threads, delay timer or size of Java
>> heap
>> >> > space. The system was operating well within its CPU and memory limits.
>> >> >
>> >> >
>> >> >
>> >> > My test is trivial: replaying XML API requests from a CSV file after
>> >> > advancing the dates in the otherwise perishable samples. I use bean
>> >> > shell to modify the dates.
>> >> >
>> >> >
>> >> >
>> >> > I allowed the script to continue labouring and finally the JVM died:
>> "GC
>> >> > overhead limit exceeded".  The solution was simply to check the "Reset
>> >> > bsh.Interpreter before each call" box on my bean shell sampler and
all
>> >> > was well.
>> >> >
>> >> >
>> >> >
>> >> > The conclusion I have drawn from this is that garbage collection
>> becomes
>> >> > a very difficult task for the JVM if the bean shell interpreter is
not
>> >> > reset, and ultimately, that is what caused the test to stall and the
>> JVM
>> >> > to die. In my experience, heap space exhaustion is the most common
>> cause
>> >> > of failures in JMeter, so having optimal garbage collection is very
>> >> > important.
>> >> >
>> >> >
>> >> >
>> >> > Therefore, I would suggest that the default "false" setting for
>> >> > resetting the bean shell interpreter in all the bean shell components
>> is
>> >> > actually not the sensible choice.  Has this default been challenged
or
>> >> > discussed previously?
>> >>
>> >> The default setting is necessary for compatibility, and cannot be
>> >> changed without potentially breaking some scripts.
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message