jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zippy Zeppoli <zippyzepp...@gmail.com>
Subject Re: complex javascript actions in jmeter load test
Date Thu, 07 Feb 2013 00:32:54 GMT
There seems to be a big misunderstanding of what is the benefit of
using a browser.
I think this thread at this point is far off topic, although it looks
like a lot of good knowledge was shaken out of it, so I will consider
it time well spent, and a question well asked.

And now the clarification:

Comparing RUM & Synthetic Page Load Times

November 14, 2012 5:30 pm | 13 Comments

Yesterday I read Etsy’s October 2012 Site Performance Report. Etsy is
one of only a handful of companies that publish their performance
stats with explanations and future plans. It’s really valuable (and
brave!), and gives other developers an opportunity to learn from an
industry leader. In this article Etsy mentions that the page load time
stats are gathered from a private instance of WebPagetest. They
explain their use of synthetically-generated measurements instead of
RUM (Real User Monitoring) data:

You might be surprised that we are using synthetic tests for this
front-end report instead of Real User Monitoring (RUM) data.  RUM is a
big part of performance monitoring at Etsy, but when we are looking at
trends in front-end performance over time, synthetic testing allows us
to eliminate much of the network variability that is inherent in real
user data. This helps us tie performance regressions to specific code
changes, and get a more stable view of performance overall.

Etsy’s choice of synthetic data for tracking performance as part of
their automated build process totally makes sense. I’ve talked to many
companies that do the same thing. Teams dealing with builds and code
regressions should definitely do this. BUT… it’s important to include
RUM data when sharing performance measurements beyond the internal
devops team.

Why should RUM data always be used when talking beyond the core team?

The issue with only showing synthetic data is that it typically makes
a website appear much faster than it actually is. This has been true
since I first started tracking real user metrics back in 2004. My
rule-of-thumb is that your real users are experiencing page load times
that are twice as long as their corresponding synthetic measurements.

>From http://www.stevesouders.com/blog/2012/11/14/comparing-rum-synthetic-page-load-times/



On Wed, Feb 6, 2013 at 4:30 PM, Zippy Zeppoli <zippyzeppoli@gmail.com> wrote:
>
> I think you may be  missing the point.
> Real load cannot be tested via HTTP interactions.
> There is no DOM rendering happening.
> I can make HTTP requests all day and it won't reflect the true response time unless it's
done through a browser.
>
> Recording a script in Jmeter proxy is trivial. Simulating *real* user load is not it
requires a browser and interactions with a web application.
>
>
> On Tue, Feb 5, 2013 at 6:51 PM, Deepak Shetty <shettyd@gmail.com> wrote:
>>
>> >Actually that does matter it cannot do JavaScript. If a request requires
>> >you need to be able to click a JavaScript button then the request will
>> >never happen.
>> The point is that what happens when the button is clicked? Assuming its a
>> server - ajax call then A HTTP call is made and some parameters are passed
>> and some values are returned. Thats whats important for the load test , not
>> the fact that javascript was executed.
>> So when you record the script , you will be the person clicking the
>> button(you are recording your actions) , JMeter will record every
>> interaction that makes a call to the server and will record this as a
>> separate HTTP request and when you run the script the same request will be
>> made as if someone clicked the button!
>>
>> You dont need to use the recorder either , you can modify the script
>> yourself.
>>
>> If the javascript didnt actually make any server side call - then it doesnt
>> matter because you dont want to load test this anyway.
>>
>> Have you actually tried this? It sounds as if you have a problem recording
>> your script and you probably have concluded that JMeter doesnt do
>> javascript (true) and hence cant test websites that do javascript/ajax
>> (false)
>>
>> >Real browser is needed
>> Not for a good deal of use cases - as many of the people on this mailing
>> list can attest too.
>>
>>
>>
>>
>>
>> On Tue, Feb 5, 2013 at 6:44 PM, Zippy Zeppoli <zippyzeppoli@gmail.com>wrote:
>>
>> > Deepak,
>> > Actually that does matter it cannot do JavaScript. If a request requires
>> > you need to be able to click a JavaScript button then the request will
>> > never happen. No request will ever be made.  Also testing true web
>> > performance requires rendering the DOM, not just initiating HTTP requests
>> > and recording the response time, rps, etc.
>> >
>> > Real browser is needed, with JavaScript, and Jmeter doesn't integrate well
>> > with this, it isn't designed for this, which is understandable. The problem
>> > is there is a gap between real browser testing (owned by third party
>> > companies) and open source tools (Jmeter). There's nothing in between for
>> > real-browser based performance testing. I could go into why, but its off
>> > topic of this list, and I'd rather spare everyone the gas.
>> >
>> > Point being, Jmeter cannot solve my problem, without some serious
>> > customization.
>> >
>> >
>> >
>> > On Tue, Feb 5, 2013 at 6:33 PM, Deepak Shetty <shettyd@gmail.com> wrote:
>> >
>> > > Hi
>> > > You are getting too caught up in the JMeter doesnt do javascript thing.
>> > In
>> > > most cases it doesnt matter.
>> > > You have a webserver that is receiving HTTP requests - whether those
>> > > requests are generated via the user clicking a link or via AJAX or via
>> > > flash is hardly relevant to the webserver. It sees HTTP requests and
>> > sends
>> > > HTTP responses.
>> > > JMeter deals with HTTP request and responses. As long as you can make the
>> > > same request that your javascript is making (which you can see via the
>> > > recording feature) , you can test it with Jmeter. The fact that the proxy
>> > > cant record javascript is irrelevant - it only needs to record the
>> > > requests. If your javascript is responsible for generating the requests
>> > in
>> > > some way , then you have to replicate that within JMeter.
>> > >
>> > > Some caveats here are
>> > > a. Your toolkit should be good when you need to parameterise(JQuery is
>> > > great , GWT sucks)
>> > > b. If you need to perform some AJAX requests in parallel then JMeter isnt
>> > > quite there yet.
>> > >
>> > > Its probably true that it is easier writing scripts for Selenium like
>> > tools
>> > > - but they also need more resources as well as given how often a UI
>> > changes
>> > > as opposed to the HTTP request/response pair , sometimes JMeter scripts
>> > are
>> > > more resilient to change.
>> > >
>> > > regards
>> > > deepak
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Tue, Feb 5, 2013 at 5:29 PM, Zippy Zeppoli <zippyzeppoli@gmail.com
>> > > >wrote:
>> > >
>> > > > Proxy won't work for clicking on JavaScript.
>> > > > Selenium will, however, Selenium isn't designed for performance
>> > testing,
>> > > > with the exception of BrowserMob, who has perfected it.
>> > > >
>> > > > If JMeter had some kind of (good) Selenium integration then it might
be
>> > > > able to achieve it.
>> > > >
>> > > > I think JMeter is a good tool, but it's not built for this kind of
>> > work.
>> > > >
>> > > > On Tue, Feb 5, 2013 at 5:20 PM, David Luu <mangaroo@gmail.com>
wrote:
>> > > >
>> > > > > I'm not familiar with the full feature set of BrowserMob, but
JMeter
>> > > can
>> > > > do
>> > > > > what you want, easy or hard, depending on your needs.
>> > > > >
>> > > > > If all you need is simple record & playback, JMeter proxy
is best
>> > > option.
>> > > > > Just record w/ proxy, it generates a basic test plan that you
can
>> > save
>> > > to
>> > > > > file and run w/ X threads to generate load, etc.
>> > > > >
>> > > > > But that approach will always use the same user, same data in
the
>> > AJAX
>> > > > > calls during load generation.
>> > > > >
>> > > > > If you need them to be unique & parameterized, that's where
it takes
>> > > some
>> > > > > work to go in and cleanup the proxy recording w/ paramaterized
data,
>> > > > > response checking (if more than just checking HTTP 200 OKs).
And
>> > you'll
>> > > > > find proxy recording generates a lot of unnecessary HTTP requests
for
>> > > > HTML,
>> > > > > CSS, image files that aren't necessary of interest as you're
testing
>> > > the
>> > > > > AJAX calls, which is where you filter out only the requests of
>> > interest
>> > > > and
>> > > > > only use those in test plan, etc.
>> > > > >
>> > > > > I believe this kind of work is needed regardless of what tool
you
>> > use,
>> > > > some
>> > > > > tools just make it easier, but there's still work to do. Not
just a
>> > one
>> > > > > touch button approach unless all you need is simple record &
>> > playback.
>> > > > >
>> > > > > On Tue, Feb 5, 2013 at 5:08 PM, Zippy Zeppoli <
>> > zippyzeppoli@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Will probably just buy BrowserMob.
>> > > > > > Too bad there isn't an open source framework to already
do this.
>> > > > > >
>> > > > > > Building this is a yak shave, and I need to be testing,
not
>> > building
>> > > a
>> > > > > test
>> > > > > > harness thats probably going to break on the next release.
>> > > > > >
>> > > > > > On Tue, Feb 5, 2013 at 3:40 PM, David Luu <mangaroo@gmail.com>
>> > > wrote:
>> > > > > >
>> > > > > > > JMeter proxy is the most integrated approach, but I
find it gives
>> > > > "too
>> > > > > > much
>> > > > > > > information", so I tend to use external tools like
browser
>> > traffic
>> > > > > > sniffers
>> > > > > > > (HttpFox, livehttpheaders, ieHttpHeaders) to see what
HTTP
>> > requests
>> > > > are
>> > > > > > > made for AJAX calls for just the requests & responses
I'm
>> > > interested
>> > > > > in,
>> > > > > > > easier to manage. But that's just my opinion, others
might find
>> > the
>> > > > > > JMeter
>> > > > > > > proxy easier to use.
>> > > > > > >
>> > > > > > > Whatever approach you take, you just need to know what
HTTP
>> > > requests
>> > > > > are
>> > > > > > > made by the AJAX calls to replicate in JMeter, and
parameterize
>> > > those
>> > > > > > > requests to take in dynamic/test data as needed, assert
>> > appropriate
>> > > > > > > response data, etc. In this case, WebDriver can be
dropped from
>> > the
>> > > > > > > equation, just gives more overhead and lowers scalability
in load
>> > > > > > > generation.
>> > > > > > >
>> > > > > > > On Tue, Feb 5, 2013 at 1:55 PM, Philippe Mouawad <
>> > > > > > > p.mouawad@ubik-ingenierie.com> wrote:
>> > > > > > >
>> > > > > > > > AjaxCall will emit a network call which JMeter
will capture.
>> > > > > > > >
>> > > > > > > > Use JMeter Server Proxy and you should have the
Ajax Calls
>> > > > recorded.
>> > > > > > > >
>> > > > > > > > Regards
>> > > > > > > > Philippe M.
>> > > > > > > >
>> > > > > > > > Follow me on twitter <https://twitter.com/philmdot>
>> > > > > > > >
>> > > > > > > > UBIK-INGENIERIE on TWITTER <https://twitter.com/ubikingenierie
>> > >
>> > > > > > > >
>> > > > > > > > UBIK LOAD PACK BLOG <http://www.ubik-ingenierie.com/blog/>
>> > > > > > > >
>> > > > > > > > On Tue, Feb 5, 2013 at 10:52 PM, Zippy Zeppoli
<
>> > > > > zippyzeppoli@gmail.com
>> > > > > > > > >wrote:
>> > > > > > > >
>> > > > > > > > > To clarify:
>> > > > > > > > > An example would be:
>> > > > > > > > > 1) log in via a form post
>> > > > > > > > > 2) look at orders in an ecommerce interface
(AJAX call)
>> > > > > > > > > 3) click on result to view order detail (AJAX)
>> > > > > > > > >
>> > > > > > > > > Pretty much a standard ecommerce transaction,
but the
>> > interface
>> > > > is
>> > > > > > all
>> > > > > > > > > javascript.
>> > > > > > > > >
>> > > > > > > > > On Mon, Feb 4, 2013 at 7:40 PM, Stott, Charlie
<
>> > > CStott@tnsi.com>
>> > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > > -----Original Message-----
>> > > > > > > > > > > From: David Luu [mailto:mangaroo@gmail.com]
>> > > > > > > > > > > Sent: Tuesday, 5 February 2013
1:18 PM
>> > > > > > > > > > > To: JMeter Users List
>> > > > > > > > > > > Subject: Re: complex javascript
actions in jmeter load
>> > test
>> > > > > > > > > > >
>> > > > > > > > > > >  >> You can use webdriver
from jmeter.  Create a
>> > webdriver
>> > > > > class
>> > > > > > > that
>> > > > > > > > > > > performs the requests and runs
the javascript via the
>> > > > browser,
>> > > > > > then
>> > > > > > > > > > run/call
>> > > > > > > > > > > it from a BSF or JSR sampler.
>> > > > > > > > > > >
>> > > > > > > > > > > Just to clarify, I take it that's
only worthwhile to do
>> > (in
>> > > > > terms
>> > > > > > > of
>> > > > > > > > > > > scalability) when using PhantomJSDriver
or HtmlUnitDriver
>> > > or
>> > > > > > > > > > FirefoxDriver
>> > > > > > > > > > > (on Linux with xvfb) with JMeter
this way?
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > "Worthwhile" depends on assumptions.
>> > > > > > > > > >
>> > > > > > > > > > > Because otherwise, the browser
GUI is the scalability
>> > > > limiting
>> > > > > > > factor
>> > > > > > > > > > even
>> > > > > > > > > > > with JMeter and Grid deployment,
and in that case, no
>> > > > > difference
>> > > > > > in
>> > > > > > > > > using
>> > > > > > > > > > > WebDriver outside JMeter to do
performance tests except
>> > if
>> > > > one
>> > > > > > > wants
>> > > > > > > > > the
>> > > > > > > > > > > JMeter logging/reporting facilities
to help performance
>> > > test,
>> > > > > > > because
>> > > > > > > > > > there's
>> > > > > > > > > > > no or minimal scalabiity gain.
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Seems the poster is more at the capability/viability
stage
>> > of
>> > > > > > > > developing
>> > > > > > > > > > tests?  We would need much more information
to start
>> > advising
>> > > > on
>> > > > > > the
>> > > > > > > > > entire
>> > > > > > > > > > load testing process, start to finish,
and what challenges
>> > > may
>> > > > > face
>> > > > > > > the
>> > > > > > > > > > poster along the way.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > On Mon, Feb 4, 2013 at 4:42 PM,
Stott, Charlie <
>> > > > > CStott@tnsi.com>
>> > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > You can use webdriver from
jmeter.  Create a webdriver
>> > > > class
>> > > > > > that
>> > > > > > > > > > > > performs the requests and
runs the javascript via the
>> > > > > browser,
>> > > > > > > then
>> > > > > > > > > > > > run/call it from a BSF or
JSR sampler.
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > > -----Original Message-----
>> > > > > > > > > > > > > From: Zippy Zeppoli [mailto:zippyzeppoli@gmail.com]
>> > > > > > > > > > > > > Sent: Tuesday, 5 February
2013 9:28 AM
>> > > > > > > > > > > > > To: JMeter Users List
>> > > > > > > > > > > > > Subject: Re: complex
javascript actions in jmeter
>> > load
>> > > > test
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The problem is Selenium
has no performance testing
>> > > > harness.
>> > > > > > > > > > > > > Sucks that it seems BrowserMob
(paid solution) is the
>> > > > only
>> > > > > > > solid
>> > > > > > > > > > option.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Until someone builds
something with Phantom.js, but
>> > it
>> > > > > seems
>> > > > > > > > JMeter
>> > > > > > > > > > > > > isn't going to cut it
here.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Fri, Feb 1, 2013 at
5:40 PM, David Luu <
>> > > > > > mangaroo@gmail.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > You'll need to figure
out what the complex
>> > javascript
>> > > > > does.
>> > > > > > > > Does
>> > > > > > > > > > > > > > it make any AJAX
requests, or is it all local
>> > client
>> > > > side
>> > > > > > > > > > > > processing/rendering?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > If it's all local,
then there's no point testing it
>> > > > with
>> > > > > > > > JMeter,
>> > > > > > > > > > > > > > that's client side
browser testing better done with
>> > > > > > Selenium.
>> > > > > > > > It
>> > > > > > > > > > > > > > won't impact the
server side load test (except
>> > delay
>> > > in
>> > > > > > > server
>> > > > > > > > > > > > > > response time for
fetching files will impact the
>> > > > > javascript
>> > > > > > > > > > > > > > execution on client
side, but that can be
>> > compensated
>> > > > w/
>> > > > > > > JMeter
>> > > > > > > > > > > > > > load test against
server with 1+ Selenium test
>> > > running
>> > > > at
>> > > > > > > same
>> > > > > > > > > > > > > > time to gauge client
side performance of site/app
>> > in
>> > > > > > > browser).
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > If the javascript
does execute AJAX requests, you
>> > > need
>> > > > to
>> > > > > > > > figure
>> > > > > > > > > > > > > > out the HTTP requests
made and mimic that in JMeter
>> > > as
>> > > > > part
>> > > > > > > of
>> > > > > > > > > your
>> > > > > > > > > > > test.
>> > > > > > > > > > > > > > You can get that
reading dev/design docs, or
>> > reverse
>> > > > > > > > > > > > > > engineer/traffic
sniffing the app while doing
>> > manual
>> > > > > > testing.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Fri, Feb 1, 2013
at 1:45 PM, Zippy Zeppoli
>> > > > > > > > > > > > > > <zippyzeppoli@gmail.com
>> > > > > > > > > > > > > > >wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Hello,
>> > > > > > > > > > > > > > > If I have a
website which requires logging in,
>> > and
>> > > > > > > executing
>> > > > > > > > > > > > > > > complex javascript
actions, how would I do this
>> > (if
>> > > > at
>> > > > > > all)
>> > > > > > > > in
>> > > > > > > > > > jmeter?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > I've heard
of writing groovy scripts to do this
>> > but
>> > > > > this
>> > > > > > > > sounds
>> > > > > > > > > > > > > > > like a
>> > > > > > > > > > > > > > lot
>> > > > > > > > > > > > > > > of work / maintenance.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Thank you.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > >
>> > > > ---------------------------------------------------------------------
>> > > > > > > > > > > > To unsubscribe, e-mail:
>> > > user-unsubscribe@jmeter.apache.org
>> > > > > > > > > > > > For additional commands, e-mail:
>> > > > user-help@jmeter.apache.org
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > > > > > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > > > > > > > > > For additional commands, e-mail:
>> > user-help@jmeter.apache.org
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message