jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oliver Erlewein (DSLWN)" <Oliver.Erlew...@datacom.co.nz>
Subject RE: Converting XML-JTL to CVS in python
Date Thu, 07 Feb 2008 03:22:12 GMT
Hi Sonam,

Done. And I think the rest is answered by sebb's post.

Cheers Oliver

-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au] 
Sent: Thursday, 7 February 2008 1:29 p.m.
To: JMeter Users List
Subject: RE: Converting XML-JTL to CVS in python

Hi Oliver - Thanks: this looks pretty valuable - do you mind adding it
to the Log analysis section of the JMeter Wiki:? 

	http://wiki.apache.org/jakarta-jmeter/LogAnalysis 

Some entries on the existing wiki page deal with JTL output format
version 2.0. The log format your code deals with is different - can you
mention that? 

If it's not too much work, do you mind documenting your understanding of
the JMeter 2.3 log format (similar to the table there for the old
format)? 

Regards,
Sonam Chauhan
--
Corporate Express Australia Ltd. 
Phone: +61-2-93350725, Email: sonam.chauhan@ce.com.au -----Original
Message-----
From: Oliver Erlewein (DSLWN) [mailto:Oliver.Erlewein@datacom.co.nz]
Sent: Thursday, 7 February 2008 10:54 AM
To: JMeter Users List
Subject: Converting XML-JTL to CVS in python

Hi all,

Just wanted to share. Maybe somebody else has use for this.
It is a Python script that translates the current (JMeter 2.3.1) XML-JTL
output to CVS. It also translates the timestamp to something readable.
And the JTL results can be filtered by a regular expression. It is
somewhat input tolerant so most Simple Data Listeners should work. 

Use is: programname.py <JTL-inFile> <CVS-outFile> <Regex filter>

The minimum regex filter should be e.g. "(.*)<httpSamp(.*)" to filter
all http samples. 

Cheers Oliver

P.S. I'm no developer so please excuse if this is not the best of breed.
I am interested if there are any cool things to make this easier code.

======================= Code Start =========================

#!/usr/bin/python

"""
Description : Split JTL file into a comma delimited CVS
by          : Oliver Erlewein (c)2008
Date        : 04.02.2008
Lang        : Python 2.4+

JMeter XML field contents:

Attribute & Content
by      Bytes
de      Data encoding
dt      Data type
ec      Error count (0 or 1, unless multiple samples are aggregated)
hn      Hostname where the sample was generated
lb      Label
lt      Latency = time to initial response (milliseconds) - not all
samplers support this
na      Number of active threads for all thread groups
ng      Number of active threads in this group
rc      Response Code (e.g. 200)
rm      Response Message (e.g. OK)
s       Success flag (true/false)
sc      Sample count (1, unless multiple samples are aggregated)
t       Elapsed time (milliseconds)
tn      Thread Name
ts      timeStamp (milliseconds since midnight Jan 1, 1970 UTC)
"""

import sys
import string
import re
import datetime
import time

startTime = time.time()
cnt                     = 0
cnt2                    = 0
failCnt                 = 0
reCompile = re.compile("\s([^\s]*?)=\"(.*?)\"")
delimiterCharacterOut = ","
lineDelimiter   = " "

def writeCSVLine(line):
    x = reCompile.findall(line)
    a = dict((row[0], row[1]) for row in x)
    try:
        a['ts'] = str(int(int(a['ts'])/1000))
        x = str(datetime.datetime.fromtimestamp(float(a['ts'])))[0:19]
        b = a['ts'] + ",\"" + x + "\"," + a['t'] + "," + a['lt'] + ",\""
+ a['s'] + "\",\"" + a['lb'] + "\"," + a['rc'] + ",\"" + a['rm'] +
"\",\"" + a['tn'] + "\",\"" + a['dt'] + "\"," + a['by'] + "\n"
    except:
        return -1
    o.write(b)
    return 1

print "Splitting JTL file"

try:
    runArgv             = sys.argv                      # Save the
command line
    jtlInfile           = str(sys.argv[1])              # Name of JTL
input file
    cvsOutfile          = str(sys.argv[2])              # Name of CVS
output file
    reFilter            = str(sys.argv[3])              # Filter the
labels (lb) for the filter
except:
    print "Error: Input format: <input file> <output file> <Filter by
regularexpression>"
    raise

try:
    f = open(jtlInfile, "r")
    o = open(cvsOutfile, "w")
except:
    raise

print "Filtering on regular expression : " + reFilter cmpFilter =
re.compile(reFilter)

for line in f:
    try:
        if cmpFilter.search(line):
           returnVal = writeCSVLine(line)
           if returnVal < 0:
              failCnt += 1
           else:
              cnt2 += 1
    except:
        print 'Error in line : ', cnt, line
        raise

    cnt += 1

endTime = time.time()
print "Time taken                    : ", str(endTime-startTime)
print "Lines processed              : ", cnt
print "Lines that passed the filter : ", cnt2
print "Lines skipped (error?)       : ", failCnt

f.close()
o.close()

======================= Code End =========================

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


The information contained in this email and any attached files are
strictly private and confidential. This email should be read by the
intended addressee only.  If the recipient of this message is not the
intended addressee, please call Corporate Express Australia Limited on
+61 2 9335 0555 or Corporate Express New Zealand Limited on +64 9 279
2555 and promptly delete this email and any attachments.  The intended
recipient of this email may only use, reproduce, disclose or distribute
the information contained in this email and any attached files with
Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or
distributing the information contained in this email and any attached
files.  Corporate Express advises that this email and any attached files
should be scanned to detect viruses. Corporate Express accepts no
liability for loss or damage (whether caused by negligence or not)
resulting from the use of any attached files.

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Mime
View raw message