jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject CSV file iteration in remote tests
Date Mon, 30 Dec 2013 14:11:22 GMT
Hello,
As dev team didn't reach consensus on commiting the piece of code to JMeter
core, it has been contributed to third party jmeter-plugins project, it
should be in next version of the library.
For those interested :
Pull request:
 - https://github.com/undera/jmeter-plugins/pull/22<https://github.com/undera/jmeter-plugins/pull/22>

- Developer Snapshots download:
-
http://jmeter-plugins.org/downloads/file/nightly/jmeter-plugins-extras-libs-1.1.2-2013-12-30_06-16.zip


Regards
Philippe M.
@philmdot

On Thursday, November 7, 2013, Oliver Lloyd wrote:

> I really like the idea of JMeter having the ability to use a central repo
> of data via something like Redis. It solves not only the problem of
> distributing unique data over multiple remote servers but also the problem
> of data that can only be used once. I see Sebb's point though that this
> solution will only help where the slaves have internet access - not always
> the case. But it is the case more and more; at least in my experience, that
> cloud-based hardware is the way to go so it makes sense to develop
> solutions that support this.
>
> I think though that such an implementation should ideally abstract a lot
> of the hassle from the user. If it would accept a csv file and a URI
> connection string, handle the import and then internalise the methods to
> pop / read the data, only exposing to the user some vars like ${pop_email}
> or ${read_email}, then that would be the perfect. That way I could enter my
> redis URI, pass a file ref to my csv and then know that each time the var
> ${pop_email} is resolved I would have one less email in my datastore -
> pretty cool.
>
>
> On 7 Nov 2013, at 06:47, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
>
> > you might be interested in this discussion:
> > -
> >
> http://mail-archives.apache.org/mod_mbox/jmeter-dev/201310.mbox/%3cCAH9fUpZtWGR7nMhpCAROoKhsRhCGWLhksPb9CrD=8GVsRYAEsw@mail.gmail.com%3e
> >
> >
> >
> > On Thursday, November 7, 2013, Oliver Erlewein wrote:
> >
> >> Oooh! I like that idea! That would also make it easier for me to update
> >> CSVs as they don't need to be distributed to the clients anymore! Would
> >> mean a bit of work but would be totally worth it.
> >>
> >>
> >>
> >> On 7 November 2013 11:46, Tim Koopmans <tim@flood.io <javascript:;>>
> >> wrote:
> >>
> >>> We approach this problem slightly differently, albeit we use an
> external
> >>> process ..
> >>>
> >>> Users upload a CSV which we bulk import into redis. Users can then make
> >>> HTTP requests to that data set via a performant API (webdis in front of
> >>> redis). This allows calls like SPOP (remove a random member) or
> >> SRANDMEMBER
> >>> (retrieve random member). This takes care of the headache of trying to
> >>> carve up CSVs across your load generators[1]
> >>>
> >>> A little bit more effort, but may give you some ideas.
> >>>
> >>> Regards,
> >>>
> >>> Tim Koopmans
> >>>
> >>> <https://flood.io>
> >>>
> >>> Level 27, 101 Collins Street
> >>> Melbourne, Vic 3000
> >>>
> >>> [1] https://flood.io/blog/9-sharing-test-data
> >>>
> >>>
> >>>
> >>> On Thu, Nov 7, 2013 at 9:36 AM, Oliver Erlewein <oliver@erlewein.net
> <javascript:;>
> >>> wrote:
> >>>
> >>>> Hi Sergio,
> >>>>
> >>>> Sorry forgot to say that that isn't a good option for me. I thought
> >> about
> >>>> it but I dynamically build remote agents off a standard build image
> and
> >>>> that makes my start procedure immensely difficult as I'd need to split
> >> the
> >>>> files and copy them to the clients before the test. I also don't have
> a
> >>>> standard number of clients so the splits are different. If I do that
I
> >>>> could also just re-randomize my files too.
> >>>>
> >>>> Cheers Oliver
> >>>>
> >>>>
> >>>> On 7 November 2013 11:28, Sergio Boso <sergio@bosoconsulting.it
> <javascript:;>>
> >> wrote:
> >>>>
> >>>>> Il 06/11/2013 23.20, Oliver Erlewein ha scritto:
> >>>>>
> >>>>> Hi all,
> >>>>>>
> >>>>>> I'm sure that this is a common problem for those using JMeter
> >>>> executions
> >>>>>> across several machines. Can't really find any solution to this.
So
> >>>> here
> >>>>>> goes:
> >>>>>>
> >>>>>> I have a plan that looks something like this:
> >>>>>>
> >>>>>> Test Plan
> >>>>>>   |-- Thread Group
> >>>>>>          |--CSV Dataset
> >>>>>>          |--HTTP Sampler (login)
> >>>>>>          |--....
> >>>>>>
> >>>>>> If I remotely distribute this all remotes will 1st start with
line
> >> one
> >>>> of
> >>>>>> the CSV file. In my case this will cause locking in the application,
> >>>>>> thereby destroying the test. Ideally I'd like to give the CSV
file a
> >>>>>> random
> >>>>>> offset for each remote client, so that it would start iterating
at
> >>>> various
> >>>>>> points in the CSV. This is not quite safe but should give me
enough
> >>>>>> variance so that the chance of locking would be minimal.
> >>>>>>
> >>>>> The only way I have found to cope with this is to manually split
the
> >> CSV
> >>>>> file, and copy each of these parts to each remote system, so that
> each
> >>>>> system uses its own set of lines.
> >>>>>
> >>>>> I'm looking forward to see if there is a better system
> >>>>> regards
> >>>>>
> >>>>> Sergio
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Ing. Sergio Boso
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message