jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shmuel Krakower <shmul...@gmail.com>
Subject Re: Web Driver based Sampler?
Date Mon, 07 May 2012 12:30:55 GMT
Hi CP,
After looking at the sampler - one question in mind:
I might want to get more details on specific action inside the script, i.e:
browse into google.com
search for a value

I'd like to know how much time the search took - and I can't find an easy
solution for this.
Any ideas?

Regards,
Shmuel.

On Mon, Apr 23, 2012 at 3:38 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com>wrote:

> Hi Shmuel,
>
> I've yet to try the WebSampler implementation on our test environment.
>  Unfortunately I'm away for the next 2 weeks, but when I get back to work I
> can try out the WebSampler and see if I can replicate the statistics I'm
> getting in our production environment.
>
> regards,
> CP
>
> On 22 April 2012 23:49, Shmuel Krakower <shmulikk@gmail.com> wrote:
>
> > I'd suggest to HP to drop C language scripting in order to make scripting
> > easier / shorter.
> >
> > As Chaitanya said - browser response times is overrated.
> >
> > Anyhow, I agree that implementing load tests with hundreds / thousands of
> > virtual users with that kind of a sampler(which running a browser) will
> > require much more hardware but as CP said, with today's cloud services
> this
> > is very affordable, thus this is not the issue.
> > I think that there are services on the web which already doing this (
> > https://browsermob.com/website-load-testing-features), but I never tried
> > those.
> >
> > CP, if you get different numbers between you monitoring solution and
> JMeter
> > reports, you should try to understand why that happens.
> > It might be for many reasons. Did developing this sampler resolved your
> > original problem? Do you get more close numbers now?
> >
> > Shmuel.
> >
> >
> > On Sun, Apr 22, 2012 at 5:27 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> > >wrote:
> >
> > > Hi Chaitanya,
> > >
> > > I guess I am looking at this from a browser view rather than from the
> > > server side.  What I tend to find is that current load testing does not
> > > properly reflect the browser load times on our site.  The reports from
> > > Google (in webmasters tools) and newrelic.com give a very different
> > > picture
> > > from that done during load/performance testing.
> > >
> > > We use HP Load Runner for our in-house load testing and the numbers we
> > get
> > > (even when we download 'all' external resources) does not come close to
> > the
> > > numbers reported by the other tools.  I guess I am attempting to
> address
> > a
> > > gap that I see in our in-house load/performance testing, and the
> > > integration of webdriver into JMeter could solve this problem.  At this
> > > stage it is an attempt to try to provide a means to simulate what our
> > > production servers see (i.e. using a real browser that will spend time
> > > executing JS/CSS).
> > >
> > > Yes, I agree that the complexity and cost of running a browser instance
> > is
> > > higher than using the standard HTTP Sampler, but with modern services
> > such
> > > as Amazon AWS, I would think that these problems would be
> > > addressable/solvable.
> > >
> > > regards,
> > > CP
> > >
> > > On 22 April 2012 06:30, chaitanya bhatt <bhatt.chaitanya@gmail.com>
> > wrote:
> > >
> > > > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <
> > cheenpin.lim@gmail.com
> > > > >wrote:
> > > > >Once you have more load
> > > > >applied to the servers/services different characteristics tend to
> > > emerge,
> > > > >which is why I was thinking that the use of JMeter would be useful.
> > > >
> > > > *I may totally sound cynical while saying this but honestly the
> reason
> > > why
> > > > I am criticising your idea is mainly due to the week rational behind
> > your
> > > > motivation to create this feature:
> > > >
> > > > 1. First of all browser induced latency on the total workload is
> > grossly
> > > > overrated. Having said that, all that an accurate simulation of
> browser
> > > > latency would do to your workload is that it would marginally reduce
> > the
> > > > total hits made on the server, which I think is bad from a
> fundamental
> > > load
> > > > testing perspective.
> > > > 2.To accommodate in-memory containers to hold 100s-1000s of virutal
> > user
> > > > browser sessions is practically impossible without testers having to
> > > > significantly size up their test lab resources. Think about the cost
> > > > factor.
> > > > 3. Looking at todays trend of browser adoption your web driver would
> > have
> > > > to be equipped with simulation capability multiple types of browsers
> > such
> > > > as IE, Chromer, Safari etc. etc. and also you would have to consider
> > > > simulating various device characteristics on the generated load.
> Think
> > > > about the complexity.
> > > >
> > > > @Shmuel: The loadrunner protocol you are talking about is Ajax
> > > > TrueClient. HP came out with Ajax TrueClient not with an intention
> > > > of generating real browser traffic but it was mainly developed to
> > reduce
> > > > the complexity of scripting apps that employ unintelligible data
> > > > serialization like Google GWT. With this HP's protocol you wouldn't
> > have
> > > a
> > > > necessity of correlating session variables at all and there by you
> > would
> > > be
> > > > significantly reducing the time to develop scripts. However, this
> > > protocol
> > > > currently has poor user adoption due to various limitations.
> > > >
> > > > Thanks
> > > > Chaitanya M Bhatt
> > > > http://www.performancecompetence.com
> > > >
> > > > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <
> cheenpin.lim@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > The tools mentioned help a great deal, however they generally look
> at
> > > it
> > > > > from a single browser to a single server scenario.  Once you have
> > more
> > > > load
> > > > > applied to the servers/services different characteristics tend to
> > > emerge,
> > > > > which is why I was thinking that the use of JMeter would be useful.
> > > > >
> > > > > Furthermore, if our primary audience is hitting the site using a
> > > browser,
> > > > > then would it not make sense to use the same tool to simulate the
> > load?
> > > > >
> > > > > Just my thoughts.
> > > > >
> > > > > regards,
> > > > > CP
> > > > >
> > > > > On 21 April 2012 17:26, chaitanya bhatt <bhatt.chaitanya@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Pleanty of awesome tools are already available for free to meet
> > your
> > > > > goals.
> > > > > >
> > > > > > Free:
> > > > > > https://developers.google.com/pagespeed/
> > > > > > http://yslow.org/
> > > > > >
> > > > > > Free or Premium:
> > > > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx
> > > > > >
> > > > > >
> > > > > > Selenium has a webdriver which you can plan on extending - if
you
> > > will.
> > > > > > IMHO Jmeter is not the right platform for this.
> > > > > > Thanks
> > > > > > Chaitnaya M Bhatt
> > > > > > http://www.performancecompetence.com
> > > > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim <
> > > > cheenpin.lim@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > After looking at the current HTTP Sampler, I would like
to
> > propose
> > > a
> > > > > > > WebDriver based implementation.  The problem is that with
a lot
> > of
> > > > > modern
> > > > > > > websites using AJAX, advanced CSS3 (transitions/transforms)
to
> do
> > > the
> > > > > > > rendering, the performance problem may not be 'visible'
to the
> > > > standard
> > > > > > > HTTP Sampler.  The idea is that if we could use JMeter
to
> > control a
> > > > > > browser
> > > > > > > and collect the time it takes to render each 'page' it
would
> give
> > > the
> > > > > > users
> > > > > > > an understanding of how long a page would take to render.
> > > > > > >
> > > > > > > Some of the high level goals are as follows:
> > > > > > >
> > > > > > >   1. Allow any WebDriver supported browser to be used to
> perform
> > > the
> > > > > > >   Sampling.
> > > > > > >   2. Provide a BeanShell/BSF style pane where the user
can
> script
> > > the
> > > > > > >   behaviour of WebDriver.
> > > > > > >
> > > > > > > I've got a very basic implementation working here:
> > > > > > > https://github.com/cplim/JMeter - you'll have to switch
to the
> > > > > > 'webmeter'
> > > > > > > branch to see the changes.  There are limitations with
this
> proof
> > > of
> > > > > > > concept, mainly:
> > > > > > >
> > > > > > >   1. It only creates Firefox instances
> > > > > > >   2. I've only provided Javascript lang support using BSF.
> > > > > > >
> > > > > > > Some outstanding questions:
> > > > > > >
> > > > > > >   1. How to support Phone (Android/iOS) based WebDriver,
as
> these
> > > > seem
> > > > > to
> > > > > > >   expect to only connect to a single phone.
> > > > > > >
> > > > > > > I would be happy to contribute the changes back to JMeter.
>  Can I
> > > get
> > > > > > some
> > > > > > > feedback on this proposal, and what I need to do to contribute
> my
> > > > > changes
> > > > > > > back to the main JMeter codebase?
> > > > > > >
> > > > > > > regards,
> > > > > > > CP Lim
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message