httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Behlendorf <br...@hyperreal.com>
Subject Re: namelookups and databases [Brandon Long <blong@uiuc.edu>] (fwd)
Date Tue, 25 Jul 1995 21:18:16 GMT


---------- Forwarded message ----------
Subject: Re: namelookups and databases
To: new-httpd@hyperreal.com
Date: Tue, 25 Jul 1995 09:37:07 -0500 (CDT)
From: Brandon Long <blong@uiuc.edu>

Last time, Randy Terbush uttered the following other thing:
> 
> HaHa! <-(PeeWee Herman laugh)
> 
> I've been attempting to shove my log data into Postgres and am
> coming to a sobering realization. It has taken 9 hours to process
> 15,000 requests.....  As I am in the process of discovering that
> much of that time is spent doing gethostbyaddr() for each entry.
> A subsequent reload without doing lookups is ontrack to be done
> in 2 hours.  This rate would obviously create a serious backlog
> on some sites if the server was direct connected to the database.
> 
> I suppose that my gethostbyaddr() results are not being cached
> by local nameservice.  One way to improve this may be to create
> that cache in my perl program.  Any other ideas on how to improve
> this?  I am beginning to question the value of this data....

On a simple perl logging script, I noticed that it did take a long time.
Generally, this was because I was only doing lookups on the IP addresses
in the log file which HTTPd didn't resolve.  

A simple (well, kinda) way to handle this is to set up a cacheing 
nameserver on your machine (or locally).  We do this on hoohoo.
I generally flush it when it hits about 20 megs of memory, but it
sure speeds things up.

Brandon

-- 
 Brandon Long   (N9WUC)     "I think, therefore, I am confused." -- RAW
 Computer Engineering   	Run Linux '95.	It's that Easy. 
 University of Illinois    blong@uiuc.edu   http://www.uiuc.edu/ph/www/blong
		Don't worry, these aren't even my views.


Mime
View raw message