httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Richards <>
Subject Re: proxying
Date Mon, 22 Jan 1996 01:41:00 GMT
In reply to Brian Behlendorf who said
> On Sun, 21 Jan 1996, sameer wrote:
> > 	dbm flocking is *bad* when you are getting on the order of one
> > hundred thousand hits/day. I used a dbm/flock method previously for my
> > daily-traffic-limit (if I ever clean up the code enough to release
> > quality I'll upload it) system, but when I was getting 100khits/day,
> > it started to hit contention so much that things *never* suceeded.

Umm, but locking only involves a change to an internal kernel table
whereas a directory walk will at least involve lots of pathname->inode
conversion and quite probably physical disk access just to find
the page and not to actually access it. In all likelyhood a db file
will stay in the filesystem cache and involve no physical disk
access at all. No matter which way you look at it locking is more
efficient by a long way compared to directory walking. If you're
getting lots of hits locking will be far more efficient.

> Hmmm, doesn't gdbm handle locking internally?  what about berkeley db?  

Berkeley db doesn't

> And if contention was a problem, doesn't that mean the backoff algorithm 
> needed to be tuned?  Also, in general you do not need to lock a dbm file 
> to read it, unless it's a file which is likely to get changed very often.

This is true. For caching there'll likely be a lot more writing than for
authentication though so it would have to be locked a lot more often.

> > 	Relating thisto the cache discussion -- a db is fine, but you
> > would have to use a seperate process db, like an mSQLserver or
> > something. The database guy I work with tells me that databases aren't
> > good for storing large chunks of data though (i.e. the size of a news
> > article or web page).
> Exactly right.  That's why when people say "why aren't more web servers 

db isn't really a database in the way database guys might think of one. It's
just a fancy lookup table with a key/"data" pair where "data" is a pointer to
a byte string which can be of arbitrary length as long as it fits in memory.

I think it'd be worth knocking up a prototype and seeing how it performs
on data such as html pages.

  Paul Richards, Netcraft Ltd.
  Phone: 0370 462071 (Mobile), +44 1225 447500 (work)

View raw message