jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheen-Pin Lim <cheenpin....@gmail.com>
Subject Re: Web Driver based Sampler?
Date Mon, 23 Apr 2012 00:38:34 GMT
Hi Shmuel,

I've yet to try the WebSampler implementation on our test environment.
 Unfortunately I'm away for the next 2 weeks, but when I get back to work I
can try out the WebSampler and see if I can replicate the statistics I'm
getting in our production environment.

regards,
CP

On 22 April 2012 23:49, Shmuel Krakower <shmulikk@gmail.com> wrote:

> I'd suggest to HP to drop C language scripting in order to make scripting
> easier / shorter.
>
> As Chaitanya said - browser response times is overrated.
>
> Anyhow, I agree that implementing load tests with hundreds / thousands of
> virtual users with that kind of a sampler(which running a browser) will
> require much more hardware but as CP said, with today's cloud services this
> is very affordable, thus this is not the issue.
> I think that there are services on the web which already doing this (
> https://browsermob.com/website-load-testing-features), but I never tried
> those.
>
> CP, if you get different numbers between you monitoring solution and JMeter
> reports, you should try to understand why that happens.
> It might be for many reasons. Did developing this sampler resolved your
> original problem? Do you get more close numbers now?
>
> Shmuel.
>
>
> On Sun, Apr 22, 2012 at 5:27 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> >wrote:
>
> > Hi Chaitanya,
> >
> > I guess I am looking at this from a browser view rather than from the
> > server side.  What I tend to find is that current load testing does not
> > properly reflect the browser load times on our site.  The reports from
> > Google (in webmasters tools) and newrelic.com give a very different
> > picture
> > from that done during load/performance testing.
> >
> > We use HP Load Runner for our in-house load testing and the numbers we
> get
> > (even when we download 'all' external resources) does not come close to
> the
> > numbers reported by the other tools.  I guess I am attempting to address
> a
> > gap that I see in our in-house load/performance testing, and the
> > integration of webdriver into JMeter could solve this problem.  At this
> > stage it is an attempt to try to provide a means to simulate what our
> > production servers see (i.e. using a real browser that will spend time
> > executing JS/CSS).
> >
> > Yes, I agree that the complexity and cost of running a browser instance
> is
> > higher than using the standard HTTP Sampler, but with modern services
> such
> > as Amazon AWS, I would think that these problems would be
> > addressable/solvable.
> >
> > regards,
> > CP
> >
> > On 22 April 2012 06:30, chaitanya bhatt <bhatt.chaitanya@gmail.com>
> wrote:
> >
> > > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <
> cheenpin.lim@gmail.com
> > > >wrote:
> > > >Once you have more load
> > > >applied to the servers/services different characteristics tend to
> > emerge,
> > > >which is why I was thinking that the use of JMeter would be useful.
> > >
> > > *I may totally sound cynical while saying this but honestly the reason
> > why
> > > I am criticising your idea is mainly due to the week rational behind
> your
> > > motivation to create this feature:
> > >
> > > 1. First of all browser induced latency on the total workload is
> grossly
> > > overrated. Having said that, all that an accurate simulation of browser
> > > latency would do to your workload is that it would marginally reduce
> the
> > > total hits made on the server, which I think is bad from a fundamental
> > load
> > > testing perspective.
> > > 2.To accommodate in-memory containers to hold 100s-1000s of virutal
> user
> > > browser sessions is practically impossible without testers having to
> > > significantly size up their test lab resources. Think about the cost
> > > factor.
> > > 3. Looking at todays trend of browser adoption your web driver would
> have
> > > to be equipped with simulation capability multiple types of browsers
> such
> > > as IE, Chromer, Safari etc. etc. and also you would have to consider
> > > simulating various device characteristics on the generated load. Think
> > > about the complexity.
> > >
> > > @Shmuel: The loadrunner protocol you are talking about is Ajax
> > > TrueClient. HP came out with Ajax TrueClient not with an intention
> > > of generating real browser traffic but it was mainly developed to
> reduce
> > > the complexity of scripting apps that employ unintelligible data
> > > serialization like Google GWT. With this HP's protocol you wouldn't
> have
> > a
> > > necessity of correlating session variables at all and there by you
> would
> > be
> > > significantly reducing the time to develop scripts. However, this
> > protocol
> > > currently has poor user adoption due to various limitations.
> > >
> > > Thanks
> > > Chaitanya M Bhatt
> > > http://www.performancecompetence.com
> > >
> > > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > The tools mentioned help a great deal, however they generally look at
> > it
> > > > from a single browser to a single server scenario.  Once you have
> more
> > > load
> > > > applied to the servers/services different characteristics tend to
> > emerge,
> > > > which is why I was thinking that the use of JMeter would be useful.
> > > >
> > > > Furthermore, if our primary audience is hitting the site using a
> > browser,
> > > > then would it not make sense to use the same tool to simulate the
> load?
> > > >
> > > > Just my thoughts.
> > > >
> > > > regards,
> > > > CP
> > > >
> > > > On 21 April 2012 17:26, chaitanya bhatt <bhatt.chaitanya@gmail.com>
> > > wrote:
> > > >
> > > > > Pleanty of awesome tools are already available for free to meet
> your
> > > > goals.
> > > > >
> > > > > Free:
> > > > > https://developers.google.com/pagespeed/
> > > > > http://yslow.org/
> > > > >
> > > > > Free or Premium:
> > > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx
> > > > >
> > > > >
> > > > > Selenium has a webdriver which you can plan on extending - if you
> > will.
> > > > > IMHO Jmeter is not the right platform for this.
> > > > > Thanks
> > > > > Chaitnaya M Bhatt
> > > > > http://www.performancecompetence.com
> > > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim <
> > > cheenpin.lim@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > After looking at the current HTTP Sampler, I would like to
> propose
> > a
> > > > > > WebDriver based implementation.  The problem is that with a
lot
> of
> > > > modern
> > > > > > websites using AJAX, advanced CSS3 (transitions/transforms)
to do
> > the
> > > > > > rendering, the performance problem may not be 'visible' to the
> > > standard
> > > > > > HTTP Sampler.  The idea is that if we could use JMeter to
> control a
> > > > > browser
> > > > > > and collect the time it takes to render each 'page' it would
give
> > the
> > > > > users
> > > > > > an understanding of how long a page would take to render.
> > > > > >
> > > > > > Some of the high level goals are as follows:
> > > > > >
> > > > > >   1. Allow any WebDriver supported browser to be used to perform
> > the
> > > > > >   Sampling.
> > > > > >   2. Provide a BeanShell/BSF style pane where the user can script
> > the
> > > > > >   behaviour of WebDriver.
> > > > > >
> > > > > > I've got a very basic implementation working here:
> > > > > > https://github.com/cplim/JMeter - you'll have to switch to the
> > > > > 'webmeter'
> > > > > > branch to see the changes.  There are limitations with this
proof
> > of
> > > > > > concept, mainly:
> > > > > >
> > > > > >   1. It only creates Firefox instances
> > > > > >   2. I've only provided Javascript lang support using BSF.
> > > > > >
> > > > > > Some outstanding questions:
> > > > > >
> > > > > >   1. How to support Phone (Android/iOS) based WebDriver, as
these
> > > seem
> > > > to
> > > > > >   expect to only connect to a single phone.
> > > > > >
> > > > > > I would be happy to contribute the changes back to JMeter. 
Can I
> > get
> > > > > some
> > > > > > feedback on this proposal, and what I need to do to contribute
my
> > > > changes
> > > > > > back to the main JMeter codebase?
> > > > > >
> > > > > > regards,
> > > > > > CP Lim
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message