httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff White" <jlw...@earthlink.net>
Subject Re: [users@httpd] Apache 1.3.27 on WinXP - Generating statistics in a GUI?
Date Thu, 13 May 2004 16:40:53 GMT

> >>
> >> Microsoft Log Parser 2.0
> >>
http://www.microsoft.com/downloads/results.aspx?freetext=log+parser
> >>
> >> IIS 6.0 Resource Kit Tools (Log Parser 2.1)
> >>
>
http://www.microsoft.com/downloads/details.aspx?familyid=56fc92ee-a71a-4c73-b628-ade629c89499&displaylang=en
>
> >>
> >> Windows Scripting LogParser
> >>
>
http://www.microsoft.com/technet/community/scriptcenter/logs/logparser/default.mspx
>
> >>
> >> TechNet Webcast: IIS Data Mining with Log Parser 2.X - Level 300
> >>
>
http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032248202&Culture=en-US
>
>
> >Will an IIS log parser parse Apache's log files?

Of course

The following examples are shown
in multiple lines for easy showing,
but when using LogParser.exe, it
should be one long line in a command
prompt window:

Example:

LogParser.exe
"SELECT Index, Text
FROM C:\Program\u0020Files\Apache\u0020Group\Apache2\conf\httpd.conf
WHERE Text LIKE '%Server%'
AND Text NOT LIKE '%#%'"
-i:TEXTLINE
-o:NAT

Real usage:

C:\>LogParser.exe "SELECT Index, Text FROM
C:\Program\u0020Files\Apache\u0020Group\Apache2\conf\httpd.conf WHERE
Text LIKE '%Server%' AND Text NOT LIKE '%#%'" -i:TEXTLINE -o:NAT

Next Example:

To get the Apache log field names:

LogParser.exe -h -i:NCSA access.log

Returns:

Input format: NCSA Common Log Format (NCSA)
Format of the NCSA Common Log Format log files

>From entity:
  <filename> | <SiteID> [, <SiteID> ... ]
  <SiteID> = '<' SiteID '>'
  SiteID can be a SiteID number, a fully qualified ADSI Path
  (e.g. "//GABRIEGI1/W3SVC/1"), or a Site name (e.g.
"www.microsoft.com")

Parameters:
 -icodepage <codepage_ID> : Input codepage [default value=guess from
                            filename and/or LogInUTF8 property]

Fields:
LogFilename (S)  LogRow (I)    RemoteHostName (S)  RemoteLogName (S)
UserName (S)     DateTime (T)  Request (S)         StatusCode (I)
BytesSent (I)

Now to use the field names:

LogParser.exe
"SELECT RemoteHostName AS cs-host,
RemoteLogName AS cs-name, UserName AS cs-username,
TO_DATE(DateTime) AS date,
TO_TIME(DateTime) AS time,
SUBSTR(Request, 1, SUB(INDEX_OF(Request, ' '),1)) AS cs-method,
SUBSTR(Request, INDEX_OF(Request, '/'), SUB( LAST_INDEX_OF(Request, '
'), INDEX_OF(Request, '/'))) AS cs-url,
SUBSTR(Request, ADD(LAST_INDEX_OF(Request,' '),1),
SUB(LAST_INDEX_OF(Request,'\"'), ADD( LAST_INDEX_OF(Request, ' '),1)))
AS http-version,
StatusCode AS s-code,
BytesSent AS s-bytes
FROM access.log
To junk1.log"
 -i:NCSA
-o:w3c

Next Example:

LogParser
"Select *
>From access.log "
-i:NCSA

LogFilename
Log
Row RemoteHostName RemoteLogName UserName DateTime
Request
                            StatusCode BytesSent
----------------------------------------------------------------------
------ ---
--- -------------- ------------- -------- ----------------------------
 ---------
--------------------------- ---------- ---------
C:\Documents and Settings\UserName\test\access.log 1
    127.0.0.1      -             -        [06/Jun/2003:18:55:01 +0000]
"GET / HTTP/1.1"                     200        1494
C:\Documents and Settings\UserName\test\access.log 2
    127.0.0.1      -             -        [06/Jun/2003:18:55:02 +0000]
"GET /apa
che_pb.gif HTTP/1.1"        200        2326

Next Example:

LogParser
"Select *
>From access.log
WHERE StatusCode=500"
-i:NCSA

Next Example:

LogParser
"Select DateTime, Request
>From access.log
TO output1.txt
WHERE StatusCode=304"
-i:NCSA

And in output1.txt

DateTime                     Request
---------------------------- ----------------------------------
[18/Jun/2003:21:14:45 +0000] "GET /apache_pb.gif HTTP/1.1"
[20/Jun/2003:17:17:19 +0000] "GET /apache_pb2_ani.gif HTTP/1.1"
[20/Jun/2003:17:19:17 +0000] "GET /apache_pb2_ani.gif HTTP/1.1"
[20/Jun/2003:17:19:19 +0000] "GET /apache_pb2_ani.gif HTTP/1.1"
[20/Jun/2003:17:19:22 +0000] "GET /apache_pb2_ani.gif HTTP/1.1"
[20/Jun/2003:17:19:25 +0000] "GET /apache_pb2_ani.gif HTTP/1.1"

Next Example:

LogParser
"Select DateTime, Request
>From access.log
TO output1.txt
WHERE Request LIKE '%.vbs%' "
 -i:NCSA

And in output1.txt

DateTime                     Request
---------------------------- -----------------------------------------
-------
[20/Jun/2003:17:09:42 +0000] "GET /scripts/webpages/vbs/cgienv.vbs
HTTP/1.1"
[20/Jun/2003:17:31:57 +0000] "GET /scripts/webpages/VBS/cgienv.vbs?
HTTP/1.1"
[20/Jun/2003:19:47:17 +0000] "GET /scripts/webpages/vbs/cgienv.vbs
HTTP/1.1"
[21/Jun/2003:17:51:34 +0000] "GET /scripts/vbs/CgiEnv.vbs HTTP/1.1"

Next Example:

LogParser
"Select LogRow, DateTime, Request
>From access.log
TO output1.txt
WHERE Request
LIKE '%apache_pb2_ani%' "
-i:NCSA
-o:W3C

Statistics:
-----------
Elements processed: 398
Elements output:    12
Execution time:     0.03 seconds

And in output1.txt

#Software: Microsoft Log Parser
#Version: 1.0
#Date: 2003-06-29 22:02:41
#Fields: LogRow DateTime Request
174 2003-06-20 17:16:51 "GET /apache_pb2_ani.gif HTTP/1.1"
176 2003-06-20 17:17:19 "GET /apache_pb2_ani.gif HTTP/1.1"
178 2003-06-20 17:19:17 "GET /apache_pb2_ani.gif HTTP/1.1"

Next Example:

LogParser
"Select LogRow, DateTime, Request
>From access.log
TO output1.txt
WHERE Request
LIKE '%apache_pb2_ani%' "
-i:NCSA
-o:CSV

Statistics:
-----------
Elements processed: 398
Elements output:    12
Execution time:     0.03 seconds

Output1.txt

LogRow, DateTime,       Request
174,    2003-06-20 17:16:51,    "GET /apache_pb2_ani.gif HTTP/1.1"
176,    2003-06-20 17:17:19,    "GET /apache_pb2_ani.gif HTTP/1.1"
178,    2003-06-20 17:19:17,    "GET /apache_pb2_ani.gif HTTP/1.1"
180,    2003-06-20 17:19:19,    "GET /apache_pb2_ani.gif HTTP/1.1"

Next Example:

LogParser
"Select LogRow, DateTime, Request
>From access.log
TO output1.txt
WHERE Request
LIKE '%apache_pb2_ani%' "
-i:NCSA
-o:XML

Statistics:
-----------
Elements processed: 398
Elements output:    12
Execution time:     0.03 seconds

Output1.txt

 ¦< ? x m l   v e r s i o n = " 1 . 0 "   s t a n d a l o n e = " y e
s " ? >  ?

¦
 < ! D O C T Y P E   R O O T
[                                                 ¦

¦
   < ! A T T L I S T   R O O T   D A T E _ C R E A T E D   C D A T A
# R E Q ¦

¦
   < ! A T T L I S T   R O O T   C R E A T E D _ B Y   C D A T A   # R
E Q U I ¦

¦
   < ! E L E M E N T   L o g R o w   ( # P C D A T A ) >
¦

¦
   < ! E L E M E N T   D a t e T i m e   ( # P C D A T A ) >
¦

¦
   < ! E L E M E N T   R e q u e s t   ( # P C D A T A ) >
¦

¦
   < ! E L E M E N T   R O W   ( L o g R o w ,   D a t e T i m e ,   R
e q u e ¦

¦
   < ! E L E M E N T   R O O T   ( R O W * ) >
¦

¦
 ] >
¦

¦
 < R O O T   D A T E _ C R E A T E D = " 2 0 0 3 - 0 6 - 2 9   2 2 : 1
1 : 0 0 ¦

 A T E D _ B Y = " M i c r o s o f t   L o g   P a r s e r   V 2 . 1 "
>

 < R O W >

   < L o g R o w >

   1 7 4

   < / L o g R o w >

   < D a t e T i m e >

   2 0 0 3 - 0 6 - 2 0   1 7 : 1 6 : 5 1

   < / D a t e T i m e >

   < R e q u e s t >

   & q u o t ; G E T   / a p a c h e _ p b 2 _ a n i . g i f   H T T P
/ 1 .

   < / R e q u e s t >

 < / R O W >
mmand>

Next Example:

Get TOP 10 requests for this log file

LogParser
"SELECT TOP 10 Request,
COUNT(*) AS Total
FROM access.log
GROUP BY Request
ORDER BY Total
DESC"
-i:NCSA
-o:NAT

Request                                   Total
----------------------------------------- -----
"GET /cgi-bin/Asp/ApAspNet.exe HTTP/1.1"  61
"GET /cgi-bin/CgiDemo.exe HTTP/1.1"       42
"GET /scripts/vbs/CgiEnv.vbs HTTP/1.1"    30
"GET /apache_pb.gif HTTP/1.1"             23
"GET /cgi-bin/asp/ApAspNet.exe HTTP/1.1"  23
"GET /cgi-bin/asp/apaspnet.exe HTTP/1.1"  23
"POST /aspnet/testhostfile.aspx HTTP/1.1" 20
"GET /aspnet/goto.aspx HTTP/1.1"          18
"POST /aspnet/redirect.aspx HTTP/1.1"     14
"GET /apache_pb2_ani.gif HTTP/1.1"        12

Statistics:
-----------
Elements processed: 398
Elements output:    10
Execution time:     0.03 seconds

Next Example:

LogParser
"SELECT TOP 10 Request,
COUNT(*) AS Total,
AVG(BytesSent) AS AvgBytesSent
FROM access.log
GROUP BY Request
ORDER BY Total
DESC"
-i:NCSA
-o:NAT

Request                                   Total AvgBytesSent
----------------------------------------- ----- ------------
"GET /cgi-bin/Asp/ApAspNet.exe HTTP/1.1"  61    623
"GET /cgi-bin/CgiDemo.exe HTTP/1.1"       42    384
"GET /scripts/vbs/CgiEnv.vbs HTTP/1.1"    30    1609
"GET /apache_pb.gif HTTP/1.1"             23    606
"GET /cgi-bin/asp/ApAspNet.exe HTTP/1.1"  23    1414
"GET /cgi-bin/asp/apaspnet.exe HTTP/1.1"  23    1437
"POST /aspnet/testhostfile.aspx HTTP/1.1" 20    12541
"GET /aspnet/goto.aspx HTTP/1.1"          18    2773
"POST /aspnet/redirect.aspx HTTP/1.1"     14    840
"GET /apache_pb2_ani.gif HTTP/1.1"        12    180

Statistics:
-----------
Elements processed: 398
Elements output:    10
Execution time:     0.03 seconds

Next Example:

Find

ServerRoot
ServerName
Listen
LoadModule

If you want text mode only use output as
-o:NAT

LogParser.exe
"SELECT Index, Text
FROM C:\Program\u0020Files\Apache\u0020Group\Apache2\conf\httpd.conf
WHERE Text LIKE '%Server%'
AND Text NOT LIKE '%#%'"
-i:TEXTLINE
-o:NAT

Returns

Index Text
----- --------------------------------------------------
57    ServerRoot "C:/Program Files/Apache Group/Apache2"
222   ServerAdmin webmaster@localhost
236   ServerName localhost:80
480   ServerTokens Full
490   ServerSignature On

Statistics:
-----------
Elements processed: 1027
Elements output:    5
Execution time:     0.03 seconds

Next Example:

And to search Apache source code try

C:\>LogParser.exe
"SELECT LogFilename, Index, Text
FROM
c:\program\u0020files\apache\src\apache2\httpd-2.0.49\modules\hello\*.
*
WHERE Text LIKE '%ap_rputs%'
AND Text NOT LIKE '%#%'"
-i:TEXTLINE
-o:nat

LogFilename      Index Text
----------------------------------------------------------------------
----------
----- ----- ------------------------------------------
c:\program
files\apache\src\apache2\httpd-2.0.49\modules\hello\mod_hello.c 90
ap_rputs("Hello World from mod_hello" , r);

Statistics:
-----------
Elements processed: 313
Elements output:    1
Execution time:     0.17 seconds

Next Example:

File system usage:

C:\>Logparser
"Select Name,
Size
FROM C:\testfolder\*.*
ORDER BY Size
DESC"
-i:FS
-recurse:OFF

Returns:

Name             Size
---------------- ----
supporttools.lst 7834
deploy.lst       1001
sup_srv.lst      580
sup_pro.lst      511
..               0
.                0

Statistics:
-----------
Elements processed: 6
Elements output:    6
Execution time:     0.03 seconds

Next Example:

The following command creates a
CSV file containing information about
all the files larger than 500 KB in the
System32 folder:

logparser
"SELECT Name, Size, Attributes
FROM C:\winnt\system32\*.*
TO files.csv
WHERE Size>512000"
-i:FS
-o:CSV

The resulting file is similar to the following:

Name,  Size,  Attributes
adminpak.msi, 13135360, -A-------
adprop.dll, 740864, -A-------
advapi32.dll, 546304, -A-------
autochk.exe, 573952, -A-------
autoconv.exe, 587264, -A-------
autofmt.exe, 566784, -A-------

Next Example:

C:\>LogParser
"SELECT DISTINCT
REPLACE_STR('Cat is brown and', 'brown', 'yellow')
FROM test1.txt
TO output1.txt "
-i:TEXTLINE

WARNING: Output format not specified - using NAT output format.

Statistics:
-----------
Elements processed: 11
Elements output:    1
Execution time:     0.03 seconds

Last Example:

LogParser.exe
"SELECT EventTypeName, TimeGenerated, Message
FROM Application
WHERE SourceName = 'Apache Service' "
-o:NAT

Returns:

EventTypeName TimeGenerated      Message


------------- ------------------ -------------------------------------
----------
----------------------------------------------------------------------
----------
----------------------------------------------------------
Error event   8/17/2003 19:08:05 The Apache service named reported the
following
 error: >>> Syntax error on line 174 of C:/Program Files/Apache
Group/Apache2/co
nf/httpd.conf: .
Error event   8/17/2003 19:08:05 The Apache service named reported the
following
 error: >>> Can't locate API module structure `hel1_module' in file
C:/Program F
iles/Apache Group/Apache2/modules/mod_hel1.so: No error .
Error event   2/6/2004 13:42:07  The Apache service named reported the
following
 error: >>> Syntax error on line 175 of C:/Program Files/Apache
Group/Apache2/co
nf/httpd.conf: .

Task completed with warnings.
There have been 192 parse errors

Statistics:
-----------
Elements processed: 2057
Elements output:    20
Execution time:     6.44 seconds

See LogParser works with Apache!

Also with Windows Scripting all output
can be sent to IE for complete GUI usage,
or use the built-in DataGrid in 2.1
-o:DataGrid
for quick GUI usage or use LogParser
template html format files.

Jeff



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message