jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Koopmans <...@flood.io>
Subject Re: CSV file iteration in remote tests
Date Mon, 30 Dec 2013 20:05:58 GMT
Great work Philippe, thanks for contributing this plugin!

Regards,

Tim Koopmans
+61 3 9221 6309



Level 27, 101 Collins Street
Melbourne, Vic 3000




On Tue, Dec 31, 2013 at 1:11 AM, Philippe Mouawad
<philippe.mouawad@gmail.com> wrote:
> Hello,
> As dev team didn't reach consensus on commiting the piece of code to JMeter
> core, it has been contributed to third party jmeter-plugins project, it
> should be in next version of the library.
> For those interested :
> Pull request:
>  - https://github.com/undera/jmeter-plugins/pull/22<https://github.com/undera/jmeter-plugins/pull/22>
>
> - Developer Snapshots download:
> -
> http://jmeter-plugins.org/downloads/file/nightly/jmeter-plugins-extras-libs-1.1.2-2013-12-30_06-16.zip
>
>
> Regards
> Philippe M.
> @philmdot
>
> On Thursday, November 7, 2013, Oliver Lloyd wrote:
>
>> I really like the idea of JMeter having the ability to use a central repo
>> of data via something like Redis. It solves not only the problem of
>> distributing unique data over multiple remote servers but also the problem
>> of data that can only be used once. I see Sebb's point though that this
>> solution will only help where the slaves have internet access - not always
>> the case. But it is the case more and more; at least in my experience, that
>> cloud-based hardware is the way to go so it makes sense to develop
>> solutions that support this.
>>
>> I think though that such an implementation should ideally abstract a lot
>> of the hassle from the user. If it would accept a csv file and a URI
>> connection string, handle the import and then internalise the methods to
>> pop / read the data, only exposing to the user some vars like ${pop_email}
>> or ${read_email}, then that would be the perfect. That way I could enter my
>> redis URI, pass a file ref to my csv and then know that each time the var
>> ${pop_email} is resolved I would have one less email in my datastore -
>> pretty cool.
>>
>>
>> On 7 Nov 2013, at 06:47, Philippe Mouawad <philippe.mouawad@gmail.com>
>> wrote:
>>
>> > you might be interested in this discussion:
>> > -
>> >
>> http://mail-archives.apache.org/mod_mbox/jmeter-dev/201310.mbox/%3cCAH9fUpZtWGR7nMhpCAROoKhsRhCGWLhksPb9CrD=8GVsRYAEsw@mail.gmail.com%3e
>> >
>> >
>> >
>> > On Thursday, November 7, 2013, Oliver Erlewein wrote:
>> >
>> >> Oooh! I like that idea! That would also make it easier for me to update
>> >> CSVs as they don't need to be distributed to the clients anymore! Would
>> >> mean a bit of work but would be totally worth it.
>> >>
>> >>
>> >>
>> >> On 7 November 2013 11:46, Tim Koopmans <tim@flood.io <javascript:;>>
>> >> wrote:
>> >>
>> >>> We approach this problem slightly differently, albeit we use an
>> external
>> >>> process ..
>> >>>
>> >>> Users upload a CSV which we bulk import into redis. Users can then make
>> >>> HTTP requests to that data set via a performant API (webdis in front
of
>> >>> redis). This allows calls like SPOP (remove a random member) or
>> >> SRANDMEMBER
>> >>> (retrieve random member). This takes care of the headache of trying
to
>> >>> carve up CSVs across your load generators[1]
>> >>>
>> >>> A little bit more effort, but may give you some ideas.
>> >>>
>> >>> Regards,
>> >>>
>> >>> Tim Koopmans
>> >>>
>> >>> <https://flood.io>
>> >>>
>> >>> Level 27, 101 Collins Street
>> >>> Melbourne, Vic 3000
>> >>>
>> >>> [1] https://flood.io/blog/9-sharing-test-data
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Nov 7, 2013 at 9:36 AM, Oliver Erlewein <oliver@erlewein.net
>> <javascript:;>
>> >>> wrote:
>> >>>
>> >>>> Hi Sergio,
>> >>>>
>> >>>> Sorry forgot to say that that isn't a good option for me. I thought
>> >> about
>> >>>> it but I dynamically build remote agents off a standard build image
>> and
>> >>>> that makes my start procedure immensely difficult as I'd need to
split
>> >> the
>> >>>> files and copy them to the clients before the test. I also don't
have
>> a
>> >>>> standard number of clients so the splits are different. If I do
that I
>> >>>> could also just re-randomize my files too.
>> >>>>
>> >>>> Cheers Oliver
>> >>>>
>> >>>>
>> >>>> On 7 November 2013 11:28, Sergio Boso <sergio@bosoconsulting.it
>> <javascript:;>>
>> >> wrote:
>> >>>>
>> >>>>> Il 06/11/2013 23.20, Oliver Erlewein ha scritto:
>> >>>>>
>> >>>>> Hi all,
>> >>>>>>
>> >>>>>> I'm sure that this is a common problem for those using JMeter
>> >>>> executions
>> >>>>>> across several machines. Can't really find any solution
to this. So
>> >>>> here
>> >>>>>> goes:
>> >>>>>>
>> >>>>>> I have a plan that looks something like this:
>> >>>>>>
>> >>>>>> Test Plan
>> >>>>>>   |-- Thread Group
>> >>>>>>          |--CSV Dataset
>> >>>>>>          |--HTTP Sampler (login)
>> >>>>>>          |--....
>> >>>>>>
>> >>>>>> If I remotely distribute this all remotes will 1st start
with line
>> >> one
>> >>>> of
>> >>>>>> the CSV file. In my case this will cause locking in the
application,
>> >>>>>> thereby destroying the test. Ideally I'd like to give the
CSV file a
>> >>>>>> random
>> >>>>>> offset for each remote client, so that it would start iterating
at
>> >>>> various
>> >>>>>> points in the CSV. This is not quite safe but should give
me enough
>> >>>>>> variance so that the chance of locking would be minimal.
>> >>>>>>
>> >>>>> The only way I have found to cope with this is to manually split
the
>> >> CSV
>> >>>>> file, and copy each of these parts to each remote system, so
that
>> each
>> >>>>> system uses its own set of lines.
>> >>>>>
>> >>>>> I'm looking forward to see if there is a better system
>> >>>>> regards
>> >>>>>
>> >>>>> Sergio
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> Ing. Sergio Boso
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message