forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Ruby" <ru...@us.ibm.com>
Subject RE: Graph data
Date Wed, 20 Feb 2002 16:35:52 GMT
John Morrison wrote:
>
> The only possible things which spring to mind would be to possibly:
>
> 1) don't group the data but repeat date information per datum
> 2) only have one source file and save doing the xinclude stuff...

Certainly can be done as a post processing step.  The data reduction of the
logs is very resource intensive.  So the question is whether this post
processing should be done in Perl, XSLT, or Java.

> Sam - is it possible to see the scripts/cvs add them?

Here's what I've got so far.

#!/usr/bin/perl
use HTTP::Date;
use POSIX "strftime";

$year = sprintf("%.4d", shift);
$month = sprintf("%.2d", shift);
$day = sprintf("%.2d", shift);
@date = localtime(str2time("$year-$month-$day"));
$week = strftime("%U", @date);
$dow = strftime("%u", @date);

$file="/x2/logarchive/www/$year/$month/$day.gz";

open OUTPUT, ">$year-$month-$day.xml";
print OUTPUT "<data group='Downloads' year='$year' week='$week'";
print OUTPUT " month='$month' day='$day' dow='$dow'>\n";

open LOG, "gzip --decompress --stdout $file |";

while (<LOG>) {
  next unless / 200 /;
  next unless /^xml.apache.org/;
  next unless /\.zip / or /\.tar[ .]/ ;
  next unless /xml.apache.org\/dist\/([-\w]*)\//;

  $count{$1}++;
}

foreach $key (sort keys %count) {
  print OUTPUT "  <datum dir='$key' value='$count{$key}'/>\n";
}

print OUTPUT "</data>\n";
close LOG;
close OUTPUT;

unlink "$day";

- Sam Ruby


Mime
View raw message