jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Lloyd <oliver_ll...@hotmail.com>
Subject Re: CSV file iteration in remote tests
Date Thu, 07 Nov 2013 14:18:45 GMT
I really like the idea of JMeter having the ability to use a central repo of data via something
like Redis. It solves not only the problem of distributing unique data over multiple remote
servers but also the problem of data that can only be used once. I see Sebb's point though
that this solution will only help where the slaves have internet access - not always the case.
But it is the case more and more; at least in my experience, that cloud-based hardware is
the way to go so it makes sense to develop solutions that support this.

I think though that such an implementation should ideally abstract a lot of the hassle from
the user. If it would accept a csv file and a URI connection string, handle the import and
then internalise the methods to pop / read the data, only exposing to the user some vars like
${pop_email} or ${read_email}, then that would be the perfect. That way I could enter my redis
URI, pass a file ref to my csv and then know that each time the var ${pop_email} is resolved
I would have one less email in my datastore - pretty cool.


On 7 Nov 2013, at 06:47, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:

> you might be interested in this discussion:
> -
> http://mail-archives.apache.org/mod_mbox/jmeter-dev/201310.mbox/%3cCAH9fUpZtWGR7nMhpCAROoKhsRhCGWLhksPb9CrD=8GVsRYAEsw@mail.gmail.com%3e
> 
> 
> 
> On Thursday, November 7, 2013, Oliver Erlewein wrote:
> 
>> Oooh! I like that idea! That would also make it easier for me to update
>> CSVs as they don't need to be distributed to the clients anymore! Would
>> mean a bit of work but would be totally worth it.
>> 
>> 
>> 
>> On 7 November 2013 11:46, Tim Koopmans <tim@flood.io <javascript:;>>
>> wrote:
>> 
>>> We approach this problem slightly differently, albeit we use an external
>>> process ..
>>> 
>>> Users upload a CSV which we bulk import into redis. Users can then make
>>> HTTP requests to that data set via a performant API (webdis in front of
>>> redis). This allows calls like SPOP (remove a random member) or
>> SRANDMEMBER
>>> (retrieve random member). This takes care of the headache of trying to
>>> carve up CSVs across your load generators[1]
>>> 
>>> A little bit more effort, but may give you some ideas.
>>> 
>>> Regards,
>>> 
>>> Tim Koopmans
>>> 
>>> <https://flood.io>
>>> 
>>> Level 27, 101 Collins Street
>>> Melbourne, Vic 3000
>>> 
>>> [1] https://flood.io/blog/9-sharing-test-data
>>> 
>>> 
>>> 
>>> On Thu, Nov 7, 2013 at 9:36 AM, Oliver Erlewein <oliver@erlewein.net<javascript:;>
>>> wrote:
>>> 
>>>> Hi Sergio,
>>>> 
>>>> Sorry forgot to say that that isn't a good option for me. I thought
>> about
>>>> it but I dynamically build remote agents off a standard build image and
>>>> that makes my start procedure immensely difficult as I'd need to split
>> the
>>>> files and copy them to the clients before the test. I also don't have a
>>>> standard number of clients so the splits are different. If I do that I
>>>> could also just re-randomize my files too.
>>>> 
>>>> Cheers Oliver
>>>> 
>>>> 
>>>> On 7 November 2013 11:28, Sergio Boso <sergio@bosoconsulting.it<javascript:;>>
>> wrote:
>>>> 
>>>>> Il 06/11/2013 23.20, Oliver Erlewein ha scritto:
>>>>> 
>>>>> Hi all,
>>>>>> 
>>>>>> I'm sure that this is a common problem for those using JMeter
>>>> executions
>>>>>> across several machines. Can't really find any solution to this.
So
>>>> here
>>>>>> goes:
>>>>>> 
>>>>>> I have a plan that looks something like this:
>>>>>> 
>>>>>> Test Plan
>>>>>>   |-- Thread Group
>>>>>>          |--CSV Dataset
>>>>>>          |--HTTP Sampler (login)
>>>>>>          |--....
>>>>>> 
>>>>>> If I remotely distribute this all remotes will 1st start with line
>> one
>>>> of
>>>>>> the CSV file. In my case this will cause locking in the application,
>>>>>> thereby destroying the test. Ideally I'd like to give the CSV file
a
>>>>>> random
>>>>>> offset for each remote client, so that it would start iterating at
>>>> various
>>>>>> points in the CSV. This is not quite safe but should give me enough
>>>>>> variance so that the chance of locking would be minimal.
>>>>>> 
>>>>> The only way I have found to cope with this is to manually split the
>> CSV
>>>>> file, and copy each of these parts to each remote system, so that each
>>>>> system uses its own set of lines.
>>>>> 
>>>>> I'm looking forward to see if there is a better system
>>>>> regards
>>>>> 
>>>>> Sergio
>>>>> 
>>>>> --
>>>>> 
>>>>> Ing. Sergio Boso
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
> 
> 
> -- 
> Cordialement.
> Philippe Mouawad.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message