jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janusz Kowalczyk <kowalczykjan...@gmail.com>
Subject Re: How to read a zipped text file and iterate over all the lines
Date Wed, 10 Jul 2013 10:31:26 GMT
btw. I know that closing & reopening a BufferedReader for the same file is
lame, but I don't have enough time to optimize it :)


On 10 July 2013 11:28, Janusz Kowalczyk <kowalczykjanusz@gmail.com> wrote:

> OK, I solved this problem by:
> adding "Only once controller" to the thread group
> putting JSR223 Sampler to that controller
> adding a "ForEach Controller" after that controller and iterating over all
> the "CONTEXT" variables.
>
> "ForEach Controller" configuration looks like:
> Input variable prefix: CONTEXT
> Start index for loop: 0
> End index for loop: ${LINE_CNT}
> Output variable name: CONTEXT
> Select "Add "_" before number?"
>
>
> One thing to remember is that when the txt file inside the zip archive is
> big, then you need to increase the heap size!
>
> *And here's the code:*
> import java.io.*;
> import java.util.zip.ZipEntry;
> import java.util.zip.ZipFile;
> import org.apache.log.Logger;
> import org.apache.jmeter.threads.JMeterVariables;
> import org.apache.jmeter.util.JMeterUtils;
> import java.io.IOException;
> import org.apache.jmeter.services.FileServer;
>
> /**
> * A Class that reads text file inside a zip archive and sets the as many
> "CONTEXT_{x}" variables as the number of lines in the text file.
> *
> * Author: Janusz Kowalczyk
> * Created: 2013-07-09
> **/
> public class JmeterZipReader {
>     private String zipFile;
>     private Logger log;
>     private JMeterVariables vars;
>
>     JmeterZipReader(String zipFile, Logger log, JMeterVariables vars){
>         this.zipFile = zipFile;
>         this.log = log;
>         this.vars = vars;
>         File f = new File(zipFile);
>         if( !f.exists()) {
>                log.error("FILE '"+zipFile+"' DOESN'T EXISTS!!!");
>                throw new IOException("FILE '"+zipFile+"' DOESN'T
> EXISTS!!!");
>         }
>     }
>
>     public void read(String textFile){
>         ZipFile zip;
>         ZipEntry ze;
>         InputStream input;
>         BufferedReader br;
>
>         try {
>             log.info("\n/******************************************\nOpening
> file " + textFile + " from zip archive: " + this.zipFile +
> "\n******************************************/");
>             zip=new ZipFile(this.zipFile);
>             ze=zip.getEntry(textFile);
>             input = zip.getInputStream(ze);
>             br = new BufferedReader(new InputStreamReader(input,
> "US-ASCII"));
>             log.info("Reading file: " + textFile);
>
>             int LINE_CNT = 0;
>             while (br.readLine() != null) LINE_CNT++;
>             // you can limit the number of rows to a specific number
>             // to avoid problems with the heap space
>             //if (LINE_CNT >= 35000) {
>             //         LINE_CNT = 35000;
>             //}
>             vars.put("LINE_CNT", Integer.toString(LINE_CNT));
>             br.close();
>             zip.close();
>             log.info("\n/******************************************\ntext
> file inside the zip archive has: " + LINE_CNT + "
> lines!!!\n******************************************/");
>
>
>             log.info("\n/******************************************\nReopeing
> the zip and csv file for setting up all the
> variables!!\n******************************************/");
>             zip=new ZipFile(this.zipFile);
>             ze=zip.getEntry(textFile);
>             input = zip.getInputStream(ze);
>             br = new BufferedReader(new InputStreamReader(input,
> "US-ASCII"));
>             String line;
>             for (int i=0; i < LINE_CNT; i++) {
>                 line = br.readLine();
>                 vars.put("CONTEXT_"+i, line);
>             }
>             br.close();
>             zip.close();
>             log.info("\n/******************************************\nZIP
> & CSV files were closed!!\n******************************************/");
>         }
>         catch (Exception e) {
>             log.error("Unhandled exception:");
>             e.printStackTrace();
>         }
>     }
> }
>
> String zipFilePath = FileServer.getFileServer().getBaseDir() +
> "/data/your_zipped_txt_file.zip";
> JmeterZipReader jzr = new JmeterZipReader(zipFilePath, log,
> ctx.getVariables());
> jzr.read("name_of_the_txt_file_inside_the_zip_file.txt");
>
>
> Cheers,
> Janusz
>
>
> On 9 July 2013 19:37, Deepak Shetty <shettyd@gmail.com> wrote:
>
>> >that I don't wan't to unzip those files,
>> The question is why do you want to do that? whats the benefit? - Again you
>> have setup and tear down threadgroups so you can do it before your test
>> runs and clean it up after its done (or as part of your build) - note you
>> have to factor in multiple threads too ..
>>
>>
>> On Tue, Jul 9, 2013 at 2:28 AM, Janusz Kowalczyk
>> <kowalczykjanusz@gmail.com>wrote:
>>
>> > The thing is that I don't wan't to unzip those files, but just read
>> > directly from them.
>> >
>> > OK, maybe easier way would be just returning a list or a map of
>> variables,
>> > so that I could iterate over it?
>> >
>> > ps. I put this strange loop just for the sake of the demo, so it doesn't
>> > read more that 10 lines at a time :)
>> >
>> >
>> >
>> > On 8 July 2013 23:35, Deepak Shetty <shettyd@gmail.com> wrote:
>> >
>> > > this.vars.put(this.outputVariable, line); ==> Always overwrites the
>> > current
>> > > value
>> > > should be the last line however unless your loop is also incorrect
>> > >
>> > > But the better way is to get your data in the form you want first ,
>> then
>> > > start the test rather than the test writing some code to read the data
>> > i.e.
>> > > unzip your files whatever before you mai n test begins (as part of
>> your
>> > > build or as a startup threadgroup )
>> > >
>> > >
>> > > On Mon, Jul 8, 2013 at 12:42 PM, Janusz Kowalczyk <
>> > > kowalczykjanusz@gmail.com
>> > > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > I'd like to create a script that would read a zipped text file, and
>> for
>> > > > example for each line (passed as variable) send a HTTP request.
>> > > >
>> > > > So far, I've written a script that works partially :)
>> > > > Partially, because sending a new line to log.info() prints out this
>> > new
>> > > > line (which is good :) ), but when I set the value of a variable to
>> > newly
>> > > > read line, then the value of that variable remains the same and is
>> > always
>> > > > equal to the first line of that text file.
>> > > >
>> > > > Please find the code below. To test it:
>> > > > - create a simple text file with just few lines of random text,
>> > > > - zip that text file
>> > > > - Add a JSR223 preprocessor & set the script language to Java
>> > > > - configure the script accordingly
>> > > > - add a HTTP sampler and add ${CONTEXT} to the "RAW Post Body" field
>> > > > - check the msg value of the http request in "View Results Tree"
>> > > >
>> > > >
>> > > > import java.io.*;
>> > > > import java.util.zip.ZipEntry;
>> > > > import java.util.zip.ZipFile;
>> > > > import org.apache.log.Logger;
>> > > > import org.apache.jmeter.threads.JMeterVariables;
>> > > >
>> > > > public class JmeterZipReader
>> > > > {
>> > > >     private String zipFile;
>> > > >     private Logger log;
>> > > >     private String outputVariable;
>> > > >     private JMeterVariables vars;
>> > > >
>> > > >     JmeterZipReader(String zipFile, Logger log, String
>> outputVariable,
>> > > > JMeterVariables vars){
>> > > >         this.zipFile = zipFile;
>> > > >         this.log = log;
>> > > >         this.outputVariable = outputVariable;
>> > > >         this.vars = vars;
>> > > >     }
>> > > >
>> > > >     // textFile is the name of the text file inside the zip file
>> > > >     public void read(String textFile){
>> > > >         ZipFile zip;
>> > > >         ZipEntry ze;
>> > > >         InputStream input;
>> > > >         BufferedReader br;
>> > > >
>> > > >         try {
>> > > >             zip=new ZipFile(this.zipFile);
>> > > >             ze=zip.getEntry(textFile);
>> > > >             input = zip.getInputStream(ze);
>> > > >             br = new BufferedReader(new InputStreamReader(input,
>> > > > "US-ASCII"));
>> > > >
>> > > >             String line;
>> > > >             int cnt = 0;
>> > > >             while((line = br.readLine()) != null && cnt <=
10) {
>> > > >                 this.vars.put(this.outputVariable, line);
>> > > >                 log.info(line);
>> > > >                 cnt += 1;
>> > > >             }
>> > > >
>> > > >             br.close();
>> > > >             zip.close();
>> > > >         }
>> > > >         catch (Exception e) {
>> > > >             log.error("Unhandled exception:");
>> > > >             e.printStackTrace();
>> > > >         }
>> > > >     }
>> > > > }
>> > > >
>> > > > JmeterZipReader jzr = new JmeterZipReader("/path/to/a/test.zip",
>> log,
>> > > > "CONTEXT", ctx.getVariables());
>> > > > jzr.read("test.csv");
>> > > >
>> > > >
>> > > >
>> > > > Many thanks,
>> > > > Janusz
>> > > >
>> > >
>> >
>>
>
>
>
>


-- 
Zapraszam na swojego foto-bloga: http://na100procentchyba.wordpress.com/
Autopodpis: Staraj się używać pola Ukryty do Wiadomości (UDW) przy
wysyłaniu wiadomości do wielu odbiorców, ograniczysz przez to
rozprzestrzenianie się spamu!
Autosignature: Try to use field BCC (blind carbon copy) when sending
message to many recepients, it will restrain spread of spam!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message